[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:49:54 dist-test-slave-r34j krb5kdc[11194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:49:54 dist-test-slave-r34j krb5kdc[11194](info): set up 2 sockets
Sep 26 13:49:54 dist-test-slave-r34j krb5kdc[11194](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 13:49:55.304736 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:49:55 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:49:55.431330 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34091
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:55.542550 11207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:55.542850 11207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:55.542894 11207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:55.546958 11207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:49:55.547014 11207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:55.547034 11207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:55.547050 11207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:49:55.547065 11207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:49:55.552364 11207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:55.553426 11207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:55.554533 11207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:55.564070 11213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:55.564023 11212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:55.564131 11207 server_base.cc:1034] running on GCE node
W20260926 13:49:55.564060 11215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:55.564839 11207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:55.565924 11207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:55.567093 11207 hybrid_clock.cc:648] HybridClock initialized: now 1790430595567067 us; error 38 us; skew 500 ppm
Sep 26 13:49:55 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430595, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:55.569940 11207 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:49:55.570883 11207 webserver.cc:460] Webserver started at http://127.10.236.254:42419/ using document root <none> and password file <none>
I20260926 13:49:55.571455 11207 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:55.571578 11207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:55.571805 11207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:55.573877 11207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee"
format_stamp: "Formatted at 2026-09-26 13:49:55 on dist-test-slave-r34j"
server_key: "9dd10d0b9d5acdfe2cce619e41bccb67"
server_key_iv: "7e4348892766ea0bd1df07f8c79cba00"
server_key_version: "encryptionkey@0"
I20260926 13:49:55.574461 11207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee"
format_stamp: "Formatted at 2026-09-26 13:49:55 on dist-test-slave-r34j"
server_key: "9dd10d0b9d5acdfe2cce619e41bccb67"
server_key_iv: "7e4348892766ea0bd1df07f8c79cba00"
server_key_version: "encryptionkey@0"
I20260926 13:49:55.578218 11207 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 13:49:55.580884 11222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:55.582327 11207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:49:55.582499 11207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee"
format_stamp: "Formatted at 2026-09-26 13:49:55 on dist-test-slave-r34j"
server_key: "9dd10d0b9d5acdfe2cce619e41bccb67"
server_key_iv: "7e4348892766ea0bd1df07f8c79cba00"
server_key_version: "encryptionkey@0"
I20260926 13:49:55.582652 11207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:55.608232 11207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:55.611902 11207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:55.612156 11207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:55.620697 11207 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:34091
I20260926 13:49:55.620712 11274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:34091 every 8 connection(s)
I20260926 13:49:55.621763 11207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:49:55.624847 11275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:49:55.628648 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11207
I20260926 13:49:55.628837 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:49:55.630618 11187 external_mini_cluster.cc:1428] Setting key b7fb2721b770e7d406e44bb46b96e14d
I20260926 13:49:55.632021 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee: Bootstrap starting.
I20260926 13:49:55.635013 11275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee: Neither blocks nor log segments found. Creating new log.
I20260926 13:49:55.636000 11275 log.cc:824] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee: Log is configured to *not* fsync() on all Append() calls
I20260926 13:49:55.638468 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee: No bootstrap required, opened a new log
I20260926 13:49:55.643008 11275 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34091 } }
I20260926 13:49:55.643326 11275 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:49:55.643399 11275 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb80c64a5ddc43e29c3b118aeb9f48ee, State: Initialized, Role: FOLLOWER
I20260926 13:49:55.643946 11275 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34091 } }
I20260926 13:49:55.644130 11275 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:49:55.644203 11275 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:49:55.644313 11275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:49:55.645640 11275 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34091 } }
I20260926 13:49:55.646040 11275 leader_election.cc:302] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb80c64a5ddc43e29c3b118aeb9f48ee; no voters:
I20260926 13:49:55.646407 11275 leader_election.cc:288] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:49:55.646574 11280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:49:55.646867 11280 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [term 1 LEADER]: Becoming Leader. State: Replica: cb80c64a5ddc43e29c3b118aeb9f48ee, State: Running, Role: LEADER
I20260926 13:49:55.647176 11280 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34091 } }
I20260926 13:49:55.647843 11275 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:49:55.648864 11281 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34091 } } }
I20260926 13:49:55.648991 11281 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [sys.catalog]: This master's current role is: LEADER
I20260926 13:49:55.649267 11282 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb80c64a5ddc43e29c3b118aeb9f48ee. Latest consensus state: current_term: 1 leader_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb80c64a5ddc43e29c3b118aeb9f48ee" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34091 } } }
I20260926 13:49:55.649365 11282 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee [sys.catalog]: This master's current role is: LEADER
I20260926 13:49:55.649681 11289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:49:55.653041 11289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:49:55 dist-test-slave-r34j krb5kdc[11194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:55.659601 11289 catalog_manager.cc:1347] Generated new cluster ID: bb05b4a2f9f24348a15475a6858b39f3
I20260926 13:49:55.659736 11289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:49:55.663293 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:55.634526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.10.236.254:34091)
0926 13:49:55.635048 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:55.635071 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:55.636401 (+ 1330us) server_negotiation.cc:407] Connection header received
0926 13:49:55.642602 (+ 6201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:55.642630 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:55.643001 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:55.643372 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:55.646743 (+ 3371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:55.648371 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:55.650229 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:55.651320 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:55.651501 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:55.656527 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:55.656532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:55.656544 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:55.656571 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:55.659005 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:55.659852 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:55.659860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:55.659867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:55.660018 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:55.660470 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:55.660474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:55.660476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:55.661057 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:55.661324 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:55.661925 (+ 601us) server_negotiation.cc:299] Negotiation successful
0926 13:49:55.662248 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":143,"threads_started":1}
I20260926 13:49:55.670876 11289 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:49:55.672031 11289 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:49:55.682703 11289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb80c64a5ddc43e29c3b118aeb9f48ee: Generated new TSK 0
I20260926 13:49:55.683465 11289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:49:55.715387 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34091
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:55.837399 11301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:55.837688 11301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:55.837736 11301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:55.841964 11301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:55.842029 11301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:55.842092 11301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:49:55.847411 11301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34091
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:55.848668 11301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:55.849754 11301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:55.859457 11307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:55.859647 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:55.859512 11309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:55.859858 11301 server_base.cc:1034] running on GCE node
I20260926 13:49:55.860318 11301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:55.861011 11301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:55.862228 11301 hybrid_clock.cc:648] HybridClock initialized: now 1790430595862212 us; error 29 us; skew 500 ppm
Sep 26 13:49:55 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430595, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:55.865381 11301 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:49:55.866353 11301 webserver.cc:460] Webserver started at http://127.10.236.193:40983/ using document root <none> and password file <none>
I20260926 13:49:55.866963 11301 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:55.867053 11301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:55.867285 11301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:55.869436 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "4878438ab09f416aa02a6dcb6d8406f2"
format_stamp: "Formatted at 2026-09-26 13:49:55 on dist-test-slave-r34j"
server_key: "445ff14a7261b0da7fa8acbf80e87bfc"
server_key_iv: "d9d4acc694a699873fdf3864d8ae1200"
server_key_version: "encryptionkey@0"
I20260926 13:49:55.870020 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "4878438ab09f416aa02a6dcb6d8406f2"
format_stamp: "Formatted at 2026-09-26 13:49:55 on dist-test-slave-r34j"
server_key: "445ff14a7261b0da7fa8acbf80e87bfc"
server_key_iv: "d9d4acc694a699873fdf3864d8ae1200"
server_key_version: "encryptionkey@0"
I20260926 13:49:55.874123 11301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:49:55.876869 11316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:55.878211 11301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:49:55.878360 11301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "4878438ab09f416aa02a6dcb6d8406f2"
format_stamp: "Formatted at 2026-09-26 13:49:55 on dist-test-slave-r34j"
server_key: "445ff14a7261b0da7fa8acbf80e87bfc"
server_key_iv: "d9d4acc694a699873fdf3864d8ae1200"
server_key_version: "encryptionkey@0"
I20260926 13:49:55.878490 11301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:55.895217 11301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:55.899557 11301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:55.899834 11301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:55.900558 11301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:49:55.901546 11301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:49:55.901628 11301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:55.901717 11301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:49:55.901775 11301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:55.912810 11301 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:44009
I20260926 13:49:55.912887 11429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:44009 every 8 connection(s)
I20260926 13:49:55.914064 11301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:49:55.922771 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11301
I20260926 13:49:55.922901 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:49:55.923205 11187 external_mini_cluster.cc:1428] Setting key 6e75db60584b9af055828695aac251d6
Sep 26 13:49:55 dist-test-slave-r34j krb5kdc[11194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430595, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:55.931157 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:55.916336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:45385 (local address 127.10.236.254:34091)
0926 13:49:55.916490 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:55.916494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:55.917326 (+ 832us) server_negotiation.cc:407] Connection header received
0926 13:49:55.919283 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:55.919287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:55.919341 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:55.919424 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:55.921028 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:55.921585 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:55.923066 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:55.923388 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:55.923540 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:55.926752 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:55.926758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:55.926763 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:55.926804 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:55.928585 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:55.929360 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:55.929364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:55.929366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:55.929455 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:55.929900 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:55.929902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:55.929903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:55.930090 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:55.930167 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:55.930926 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 13:49:55.931036 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 13:49:55.932713 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:55.916666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34091 (local address 127.10.236.193:45385)
0926 13:49:55.917135 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:55.917182 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:55.919030 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:55.919791 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:55.919800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:55.920257 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:49:55.920845 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:55.920862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:55.921733 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:55.921736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:55.922932 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:55.922945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:55.923589 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:55.923594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:55.923817 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:55.924653 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:49:55.924670 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:49:55.926546 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:49:55.928726 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:55.928736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:55.928758 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:55.929224 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:55.929570 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:55.929575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:55.929580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:55.929794 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:55.930212 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:49:55.930222 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:49:55.930628 (+ 406us) client_negotiation.cc:769] Sending connection context
0926 13:49:55.930903 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 13:49:55.931355 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:49:55.934288 11430 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34091
I20260926 13:49:55.934566 11430 heartbeater.cc:464] Registering TS with master...
I20260926 13:49:55.935092 11430 heartbeater.cc:511] Master 127.10.236.254:34091 requested a full tablet report, sending...
I20260926 13:49:55.936880 11239 ts_manager.cc:194] Registered new tserver with Master: 4878438ab09f416aa02a6dcb6d8406f2 (127.10.236.193:44009)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:49:55.938676 11239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:45385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:49:55.955925 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34091
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:56.076437 11435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:56.076754 11435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:56.076802 11435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:56.081132 11435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:56.081202 11435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:56.081295 11435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:49:56.087164 11435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34091
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:56.088347 11435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:56.089404 11435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:56.100121 11440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:56.100126 11441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:56.100271 11435 server_base.cc:1034] running on GCE node
W20260926 13:49:56.100219 11443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:56.100920 11435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:56.101537 11435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:56.102701 11435 hybrid_clock.cc:648] HybridClock initialized: now 1790430596102685 us; error 32 us; skew 500 ppm
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:56.105786 11435 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:49:56.106722 11435 webserver.cc:460] Webserver started at http://127.10.236.194:33481/ using document root <none> and password file <none>
I20260926 13:49:56.107343 11435 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:56.107437 11435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:56.107708 11435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:56.109844 11435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "88f033d3fb7942a19616d7cd142a363e"
format_stamp: "Formatted at 2026-09-26 13:49:56 on dist-test-slave-r34j"
server_key: "f50bca2dc1f6b370fb984dc37cb08a39"
server_key_iv: "d0c5a386a96bcd5f33ba40364d8958e9"
server_key_version: "encryptionkey@0"
I20260926 13:49:56.110426 11435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "88f033d3fb7942a19616d7cd142a363e"
format_stamp: "Formatted at 2026-09-26 13:49:56 on dist-test-slave-r34j"
server_key: "f50bca2dc1f6b370fb984dc37cb08a39"
server_key_iv: "d0c5a386a96bcd5f33ba40364d8958e9"
server_key_version: "encryptionkey@0"
I20260926 13:49:56.114537 11435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 13:49:56.117839 11450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:56.119217 11435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:49:56.119369 11435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "88f033d3fb7942a19616d7cd142a363e"
format_stamp: "Formatted at 2026-09-26 13:49:56 on dist-test-slave-r34j"
server_key: "f50bca2dc1f6b370fb984dc37cb08a39"
server_key_iv: "d0c5a386a96bcd5f33ba40364d8958e9"
server_key_version: "encryptionkey@0"
I20260926 13:49:56.119516 11435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:56.150118 11435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:56.153712 11435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:56.153925 11435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:56.154529 11435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:49:56.155417 11435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:49:56.155486 11435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:56.155606 11435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:49:56.155659 11435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:56.165525 11435 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33253
I20260926 13:49:56.165546 11563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33253 every 8 connection(s)
I20260926 13:49:56.166451 11435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:49:56.174067 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11435
I20260926 13:49:56.174206 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:49:56.174466 11187 external_mini_cluster.cc:1428] Setting key df21e007ebdc995ad1b267e9569aa013
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:56.180577 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:56.168379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:39355 (local address 127.10.236.254:34091)
0926 13:49:56.168525 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:56.168530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:56.169424 (+ 894us) server_negotiation.cc:407] Connection header received
0926 13:49:56.170708 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:56.170710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:56.170758 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:56.170834 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:56.172118 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.172651 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.173820 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.174034 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.174134 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:56.176873 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:56.176877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:56.176879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:56.176905 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:56.178420 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.178944 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.178948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.178950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.179032 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.179381 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.179386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.179387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.179661 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:56.179765 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:56.180287 (+ 522us) server_negotiation.cc:299] Negotiation successful
0926 13:49:56.180419 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:49:56.181411 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:56.168833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34091 (local address 127.10.236.194:39355)
0926 13:49:56.169270 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:56.169312 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:56.170499 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:56.171000 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:56.171007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:56.171369 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:49:56.171972 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:56.171986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.172827 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.172830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:56.173679 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:56.173687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.174204 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.174208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:56.174411 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:56.175051 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:49:56.175067 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:49:56.176710 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:49:56.178550 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:56.178556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:56.178573 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:56.178839 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:56.179125 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:56.179129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:56.179132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:56.179286 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:56.179772 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:49:56.179781 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:49:56.180039 (+ 258us) client_negotiation.cc:769] Sending connection context
0926 13:49:56.180248 (+ 209us) client_negotiation.cc:240] Negotiation successful
0926 13:49:56.180582 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
I20260926 13:49:56.182685 11564 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34091
I20260926 13:49:56.182977 11564 heartbeater.cc:464] Registering TS with master...
I20260926 13:49:56.183712 11564 heartbeater.cc:511] Master 127.10.236.254:34091 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:49:56.184899 11239 ts_manager.cc:194] Registered new tserver with Master: 88f033d3fb7942a19616d7cd142a363e (127.10.236.194:33253)
I20260926 13:49:56.185725 11239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:39355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:49:56.204794 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34091
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:56.323096 11569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:56.323414 11569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:56.323467 11569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:56.328161 11569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:56.328231 11569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:56.328300 11569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:49:56.333998 11569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34091
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:56.335040 11569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:56.336112 11569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:56.346438 11575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:56.346405 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:56.346415 11577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:56.346752 11569 server_base.cc:1034] running on GCE node
I20260926 13:49:56.347221 11569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:56.348013 11569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:56.349241 11569 hybrid_clock.cc:648] HybridClock initialized: now 1790430596349213 us; error 49 us; skew 500 ppm
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:56.356666 11569 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:49:56.357686 11569 webserver.cc:460] Webserver started at http://127.10.236.195:37779/ using document root <none> and password file <none>
I20260926 13:49:56.358280 11569 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:56.358372 11569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:56.358610 11569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:56.360819 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "b2154601102a49be9bf206f58d0037fb"
format_stamp: "Formatted at 2026-09-26 13:49:56 on dist-test-slave-r34j"
server_key: "9835862d71700ac70e80161231ef8956"
server_key_iv: "2f68b7c4b5bfd50010c0932ea83aabf9"
server_key_version: "encryptionkey@0"
I20260926 13:49:56.361387 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "b2154601102a49be9bf206f58d0037fb"
format_stamp: "Formatted at 2026-09-26 13:49:56 on dist-test-slave-r34j"
server_key: "9835862d71700ac70e80161231ef8956"
server_key_iv: "2f68b7c4b5bfd50010c0932ea83aabf9"
server_key_version: "encryptionkey@0"
I20260926 13:49:56.365396 11569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:49:56.368047 11584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:56.369496 11569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:49:56.369601 11569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "b2154601102a49be9bf206f58d0037fb"
format_stamp: "Formatted at 2026-09-26 13:49:56 on dist-test-slave-r34j"
server_key: "9835862d71700ac70e80161231ef8956"
server_key_iv: "2f68b7c4b5bfd50010c0932ea83aabf9"
server_key_version: "encryptionkey@0"
I20260926 13:49:56.369840 11569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:56.383404 11569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:56.387854 11569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:56.388098 11569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:56.388754 11569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:49:56.389928 11569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:49:56.389995 11569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:56.390064 11569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:49:56.390093 11569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:56.402410 11569 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:32879
I20260926 13:49:56.402434 11697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:32879 every 8 connection(s)
I20260926 13:49:56.403769 11569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:49:56.412266 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11569
I20260926 13:49:56.412441 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:49:56.412830 11187 external_mini_cluster.cc:1428] Setting key b21fac075b5a20ed24aa3c381bc5a37c
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:56.422362 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:56.406175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:59883 (local address 127.10.236.254:34091)
0926 13:49:56.406323 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:56.406329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:56.407773 (+ 1444us) server_negotiation.cc:407] Connection header received
0926 13:49:56.409243 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:56.409248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:56.409323 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:56.409433 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:56.411361 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.412386 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.414454 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.414781 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.414910 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:56.418114 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:56.418117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:56.418119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:56.418148 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:56.419816 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.420537 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.420540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.420541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.420621 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.421061 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.421067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.421069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.421264 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:56.421344 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:56.422056 (+ 712us) server_negotiation.cc:299] Negotiation successful
0926 13:49:56.422186 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:49:56.423534 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:56.406730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34091 (local address 127.10.236.195:59883)
0926 13:49:56.407590 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:56.407650 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:56.408965 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:56.409660 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:56.409669 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:56.410304 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:49:56.411153 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:56.411171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.412569 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.412574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:56.414308 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:56.414323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.414973 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.414978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:56.415273 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:56.416097 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:49:56.416117 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:49:56.417906 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:49:56.419948 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:56.419957 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:56.419979 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:56.420423 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:56.420720 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:56.420725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:56.420728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:56.420947 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:56.421378 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:49:56.421386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:49:56.421755 (+ 369us) client_negotiation.cc:769] Sending connection context
0926 13:49:56.422000 (+ 245us) client_negotiation.cc:240] Negotiation successful
0926 13:49:56.422433 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":146,"threads_started":1}
I20260926 13:49:56.424822 11698 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34091
I20260926 13:49:56.425103 11698 heartbeater.cc:464] Registering TS with master...
I20260926 13:49:56.425627 11698 heartbeater.cc:511] Master 127.10.236.254:34091 requested a full tablet report, sending...
I20260926 13:49:56.426853 11239 ts_manager.cc:194] Registered new tserver with Master: b2154601102a49be9bf206f58d0037fb (127.10.236.195:32879)
I20260926 13:49:56.427711 11239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:59883
I20260926 13:49:56.428078 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:49:56.455338 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:56.445320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.10.236.193:44009)
0926 13:49:56.445684 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:56.445694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:56.445725 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:49:56.445783 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:56.445791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:56.445989 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:56.446146 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:56.446861 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.447757 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.448590 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.449052 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.449180 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:56.451832 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:56.451841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:56.451866 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:56.451892 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:56.453513 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.453958 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.453964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.453966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.454052 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.454338 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.454342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.454345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.454601 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:56.454838 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:56.455000 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 13:49:56.455154 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":72,"threads_started":1}
W20260926 13:49:56.456372 11344 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57444
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:49:56.483217 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:56.474361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.10.236.193:44009)
0926 13:49:56.474580 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:56.474584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:56.474706 (+ 122us) server_negotiation.cc:407] Connection header received
0926 13:49:56.474824 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:56.474827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:56.474871 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:56.474960 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:56.475861 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.476433 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.477310 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:56.477528 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:56.477612 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:56.480086 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:56.480089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:56.480091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:56.480117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:56.481477 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.481985 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.481988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.481989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.482065 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:56.482347 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:56.482348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:56.482350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:56.482502 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:56.482603 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:56.482923 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 13:49:56.483025 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 13:49:56.484373 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11301
I20260926 13:49:56.490422 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11435
I20260926 13:49:56.495900 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11569
I20260926 13:49:56.501102 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11207
2026-09-26T13:49:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2266 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11712](info): set up 2 sockets
Sep 26 13:49:56 dist-test-slave-r34j krb5kdc[11712](info): commencing operation
krb5kdc: starting...
W20260926 13:49:57.551187 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:49:57 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:49:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:49:57.679970 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34381
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:34381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:57.792383 11725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:57.792695 11725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:57.792737 11725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:57.796831 11725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:49:57.796900 11725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:57.796921 11725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:57.796938 11725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:49:57.796954 11725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:49:57.802501 11725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:34381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:57.803656 11725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:57.804682 11725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:57.813689 11730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:57.813686 11731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:57.813858 11725 server_base.cc:1034] running on GCE node
W20260926 13:49:57.814044 11733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:57.814600 11725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:57.815920 11725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:57.817112 11725 hybrid_clock.cc:648] HybridClock initialized: now 1790430597817091 us; error 37 us; skew 500 ppm
Sep 26 13:49:57 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430597, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:57.820039 11725 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:49:57.820909 11725 webserver.cc:460] Webserver started at http://127.10.236.254:36477/ using document root <none> and password file <none>
I20260926 13:49:57.821436 11725 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:57.821494 11725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:57.821657 11725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:57.823869 11725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "f20f9d9e10d14afca8c814044636c99a"
format_stamp: "Formatted at 2026-09-26 13:49:57 on dist-test-slave-r34j"
server_key: "84ff82fd04d7200f6329b491433f42eb"
server_key_iv: "0f3b3a6135793a6e97d99242c62d2384"
server_key_version: "encryptionkey@0"
I20260926 13:49:57.824390 11725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "f20f9d9e10d14afca8c814044636c99a"
format_stamp: "Formatted at 2026-09-26 13:49:57 on dist-test-slave-r34j"
server_key: "84ff82fd04d7200f6329b491433f42eb"
server_key_iv: "0f3b3a6135793a6e97d99242c62d2384"
server_key_version: "encryptionkey@0"
I20260926 13:49:57.828225 11725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:49:57.830636 11740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:57.832202 11725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:49:57.832347 11725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "f20f9d9e10d14afca8c814044636c99a"
format_stamp: "Formatted at 2026-09-26 13:49:57 on dist-test-slave-r34j"
server_key: "84ff82fd04d7200f6329b491433f42eb"
server_key_iv: "0f3b3a6135793a6e97d99242c62d2384"
server_key_version: "encryptionkey@0"
I20260926 13:49:57.832463 11725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:57.847860 11725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:57.851328 11725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:57.851576 11725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:57.859035 11725 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:34381
I20260926 13:49:57.859061 11792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:34381 every 8 connection(s)
I20260926 13:49:57.860105 11725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:49:57.863101 11793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:49:57.866711 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11725
I20260926 13:49:57.866863 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:49:57.867110 11187 external_mini_cluster.cc:1428] Setting key aed5a8d72efd0a2549039ebb691568c1
I20260926 13:49:57.869145 11793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a: Bootstrap starting.
I20260926 13:49:57.871343 11793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a: Neither blocks nor log segments found. Creating new log.
I20260926 13:49:57.872123 11793 log.cc:824] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:49:57.874354 11793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a: No bootstrap required, opened a new log
Sep 26 13:49:57 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:57.878278 11793 raft_consensus.cc:348] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20f9d9e10d14afca8c814044636c99a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34381 } }
I20260926 13:49:57.878539 11793 raft_consensus.cc:374] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:49:57.878623 11793 raft_consensus.cc:729] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f20f9d9e10d14afca8c814044636c99a, State: Initialized, Role: FOLLOWER
I20260926 13:49:57.879086 11793 consensus_queue.cc:260] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20f9d9e10d14afca8c814044636c99a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34381 } }
I20260926 13:49:57.879236 11793 raft_consensus.cc:388] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:49:57.879307 11793 raft_consensus.cc:482] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:49:57.879436 11793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:49:57.880412 11793 raft_consensus.cc:504] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20f9d9e10d14afca8c814044636c99a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34381 } }
I20260926 13:49:57.880759 11793 leader_election.cc:302] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f20f9d9e10d14afca8c814044636c99a; no voters:
I20260926 13:49:57.881244 11793 leader_election.cc:288] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:49:57.881402 11798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:49:57.881650 11798 raft_consensus.cc:686] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [term 1 LEADER]: Becoming Leader. State: Replica: f20f9d9e10d14afca8c814044636c99a, State: Running, Role: LEADER
I20260926 13:49:57.882617 11793 sys_catalog.cc:564] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:49:57.882732 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:57.868589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.10.236.254:34381)
0926 13:49:57.869042 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:57.869056 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:57.869094 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:49:57.869826 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:57.869844 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:57.870183 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:57.870519 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:57.871323 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:57.872528 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:57.873722 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:57.874220 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:57.874423 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:57.877034 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:57.877039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:57.877054 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:57.877081 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:57.879122 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:57.879729 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:57.879745 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:57.879750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:57.879869 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:57.880164 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:57.880168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:57.880170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:57.880651 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:57.880984 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:57.881331 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 13:49:57.881669 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":145,"threads_started":1}
I20260926 13:49:57.882005 11798 consensus_queue.cc:237] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20f9d9e10d14afca8c814044636c99a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34381 } }
I20260926 13:49:57.885241 11799 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f20f9d9e10d14afca8c814044636c99a. Latest consensus state: current_term: 1 leader_uuid: "f20f9d9e10d14afca8c814044636c99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20f9d9e10d14afca8c814044636c99a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34381 } } }
I20260926 13:49:57.885432 11800 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f20f9d9e10d14afca8c814044636c99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20f9d9e10d14afca8c814044636c99a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34381 } } }
I20260926 13:49:57.885790 11799 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [sys.catalog]: This master's current role is: LEADER
I20260926 13:49:57.885802 11800 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a [sys.catalog]: This master's current role is: LEADER
W20260926 13:49:57.888248 11812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:49:57.888311 11812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:49:57.888394 11813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:49:57.889140 11813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:49:57.894410 11813 catalog_manager.cc:1347] Generated new cluster ID: 13b1cbbd79ed40f6a4454f23952bc479
I20260926 13:49:57.894470 11813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:49:57.907950 11813 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:49:57.909225 11813 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:49:57.921025 11813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f20f9d9e10d14afca8c814044636c99a: Generated new TSK 0
I20260926 13:49:57.922153 11813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:49:57.953984 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34381
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:58.074347 11819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:58.074749 11819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:58.074803 11819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:58.078981 11819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:58.079051 11819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:58.079113 11819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:49:58.084990 11819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:58.086146 11819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:58.087213 11819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:58.097432 11827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:58.097494 11819 server_base.cc:1034] running on GCE node
W20260926 13:49:58.097517 11824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:58.097432 11825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:58.098055 11819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:58.098774 11819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:58.099973 11819 hybrid_clock.cc:648] HybridClock initialized: now 1790430598099960 us; error 34 us; skew 500 ppm
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:58.102769 11819 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:49:58.103786 11819 webserver.cc:460] Webserver started at http://127.10.236.193:37151/ using document root <none> and password file <none>
I20260926 13:49:58.104357 11819 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:58.104420 11819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:58.104673 11819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:58.106752 11819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "9f75b7f735144f9387d4f4eb1f9c892d"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "e7fad26a000230b3cbfb3837f9c3d70f"
server_key_iv: "472deaf83ea4b5776a461e5235a212d7"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.107329 11819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "9f75b7f735144f9387d4f4eb1f9c892d"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "e7fad26a000230b3cbfb3837f9c3d70f"
server_key_iv: "472deaf83ea4b5776a461e5235a212d7"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.111145 11819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 13:49:58.113781 11834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.115101 11819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:49:58.115245 11819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "9f75b7f735144f9387d4f4eb1f9c892d"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "e7fad26a000230b3cbfb3837f9c3d70f"
server_key_iv: "472deaf83ea4b5776a461e5235a212d7"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.115361 11819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:58.126974 11819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:58.130257 11819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:58.130496 11819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:58.131114 11819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:49:58.132037 11819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:49:58.132083 11819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.132151 11819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:49:58.132191 11819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.141464 11819 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:33691
I20260926 13:49:58.141487 11947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:33691 every 8 connection(s)
I20260926 13:49:58.142417 11819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:49:58.151274 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11819
I20260926 13:49:58.151412 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:49:58.151724 11187 external_mini_cluster.cc:1428] Setting key cdd0f8402a281a99e1d1121dd3e9fd25
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:58.157467 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.144508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:59127 (local address 127.10.236.254:34381)
0926 13:49:58.144668 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.144673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.145600 (+ 927us) server_negotiation.cc:407] Connection header received
0926 13:49:58.146440 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.146442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.146490 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.146565 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:58.148046 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.148708 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.150001 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.150233 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.150326 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.153430 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:58.153433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:58.153435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:58.153458 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:58.155171 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.155799 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.155805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.155809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.155912 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.156293 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.156297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.156299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.156505 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:58.156618 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.157151 (+ 533us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.157298 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 13:49:58.158326 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.144962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34381 (local address 127.10.236.193:59127)
0926 13:49:58.145434 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.145477 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.146246 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.146782 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.146790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.147166 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:49:58.147843 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.147863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.148898 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.148901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.149863 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.149872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.150382 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.150385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.150529 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.151223 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:49:58.151240 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:49:58.153073 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:49:58.155308 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:58.155312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:58.155325 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:58.155690 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:58.156016 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:58.156019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:58.156020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:58.156168 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:58.156615 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:49:58.156622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:49:58.156900 (+ 278us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.157095 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.157456 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":115,"threads_started":1}
I20260926 13:49:58.159893 11948 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34381
I20260926 13:49:58.160156 11948 heartbeater.cc:464] Registering TS with master...
I20260926 13:49:58.160665 11948 heartbeater.cc:511] Master 127.10.236.254:34381 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:49:58.162398 11757 ts_manager.cc:194] Registered new tserver with Master: 9f75b7f735144f9387d4f4eb1f9c892d (127.10.236.193:33691)
I20260926 13:49:58.163682 11757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:59127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:49:58.183706 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34381
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:58.297037 11953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:58.297348 11953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:58.297435 11953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:58.301651 11953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:58.301746 11953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:58.301841 11953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:49:58.307298 11953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:58.308564 11953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:58.309656 11953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:58.319904 11961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:58.319895 11958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:58.319895 11959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:58.320324 11953 server_base.cc:1034] running on GCE node
I20260926 13:49:58.320789 11953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:58.321529 11953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:58.322806 11953 hybrid_clock.cc:648] HybridClock initialized: now 1790430598322779 us; error 46 us; skew 500 ppm
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:58.325810 11953 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:49:58.326663 11953 webserver.cc:460] Webserver started at http://127.10.236.194:34495/ using document root <none> and password file <none>
I20260926 13:49:58.327195 11953 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:58.327261 11953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:58.327422 11953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:58.329499 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "011626ecbb3e4c8a9c71d183e97dea77"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "bb517a8317ddf9104ff4456bbabd594e"
server_key_iv: "6bba548fe9f3eaed38d86672424cb547"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.330035 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "011626ecbb3e4c8a9c71d183e97dea77"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "bb517a8317ddf9104ff4456bbabd594e"
server_key_iv: "6bba548fe9f3eaed38d86672424cb547"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.334028 11953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 13:49:58.336773 11968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.338316 11953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:49:58.338439 11953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "011626ecbb3e4c8a9c71d183e97dea77"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "bb517a8317ddf9104ff4456bbabd594e"
server_key_iv: "6bba548fe9f3eaed38d86672424cb547"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.338619 11953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:58.356935 11953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:58.360703 11953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:58.360924 11953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:58.361649 11953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:49:58.362600 11953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:49:58.362650 11953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.362699 11953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:49:58.362758 11953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.372535 11953 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:35521
I20260926 13:49:58.372553 12081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:35521 every 8 connection(s)
I20260926 13:49:58.373497 11953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:49:58.380767 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 11953
I20260926 13:49:58.380928 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:49:58.381232 11187 external_mini_cluster.cc:1428] Setting key 917b50a93df7d33a65de6f4190977364
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:58.389691 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.375533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:53413 (local address 127.10.236.254:34381)
0926 13:49:58.375737 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.375741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.376794 (+ 1053us) server_negotiation.cc:407] Connection header received
0926 13:49:58.377734 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.377738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.377800 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.377888 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:58.379183 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.379893 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.381205 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.381517 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.381637 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.385076 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:58.385080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:58.385082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:58.385110 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:58.386911 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.387752 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.387757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.387759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.387871 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.388324 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.388330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.388335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.388622 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:58.388740 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.389338 (+ 598us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.389476 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 13:49:58.390466 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.375889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34381 (local address 127.10.236.194:53413)
0926 13:49:58.376628 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.376677 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.377490 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.378064 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.378073 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.378456 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:49:58.379005 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.379020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.380052 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.380054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.381075 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.381085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.381731 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.381736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.381980 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.382795 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:49:58.382816 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:49:58.384773 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:49:58.387061 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:58.387066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:58.387080 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:58.387475 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:58.388020 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:58.388022 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:58.388024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:58.388177 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:58.388754 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:49:58.388760 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:49:58.389060 (+ 300us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.389229 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.389512 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:49:58.391945 12082 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34381
I20260926 13:49:58.392184 12082 heartbeater.cc:464] Registering TS with master...
I20260926 13:49:58.392652 12082 heartbeater.cc:511] Master 127.10.236.254:34381 requested a full tablet report, sending...
I20260926 13:49:58.393819 11757 ts_manager.cc:194] Registered new tserver with Master: 011626ecbb3e4c8a9c71d183e97dea77 (127.10.236.194:35521)
I20260926 13:49:58.394433 11757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:53413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:49:58.411931 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34381
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:49:58.529213 12087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:49:58.529502 12087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:49:58.529549 12087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:49:58.533792 12087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:49:58.533866 12087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:49:58.533931 12087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:49:58.539964 12087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:49:58.541059 12087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:49:58.542161 12087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:49:58.552245 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:49:58.552228 12095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:58.552412 12087 server_base.cc:1034] running on GCE node
W20260926 13:49:58.552228 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:49:58.552917 12087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:49:58.553579 12087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:49:58.554762 12087 hybrid_clock.cc:648] HybridClock initialized: now 1790430598554732 us; error 40 us; skew 500 ppm
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:49:58.557768 12087 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:49:58.558774 12087 webserver.cc:460] Webserver started at http://127.10.236.195:36631/ using document root <none> and password file <none>
I20260926 13:49:58.559356 12087 fs_manager.cc:359] Metadata directory not provided
I20260926 13:49:58.559417 12087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:49:58.559675 12087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:49:58.561764 12087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "84491570eb544dafbdaf4aad89cac87e"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "873e3a2ad398046135a425ab222360ca"
server_key_iv: "5f7ae85d5946d26416c3592eed62b152"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.562412 12087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "84491570eb544dafbdaf4aad89cac87e"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "873e3a2ad398046135a425ab222360ca"
server_key_iv: "5f7ae85d5946d26416c3592eed62b152"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.566267 12087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260926 13:49:58.568823 12102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.570150 12087 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:49:58.570305 12087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "84491570eb544dafbdaf4aad89cac87e"
format_stamp: "Formatted at 2026-09-26 13:49:58 on dist-test-slave-r34j"
server_key: "873e3a2ad398046135a425ab222360ca"
server_key_iv: "5f7ae85d5946d26416c3592eed62b152"
server_key_version: "encryptionkey@0"
I20260926 13:49:58.570425 12087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:49:58.590628 12087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:49:58.593986 12087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:49:58.594219 12087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:49:58.594837 12087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:49:58.595760 12087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:49:58.595829 12087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.595901 12087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:49:58.595952 12087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:49:58.604903 12087 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:41769
I20260926 13:49:58.604923 12215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:41769 every 8 connection(s)
I20260926 13:49:58.605806 12087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:49:58.609068 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 12087
I20260926 13:49:58.609194 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:49:58.609476 11187 external_mini_cluster.cc:1428] Setting key ad141000f9b22e4b1f8e0f8108094ae0
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:49:58.619980 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.607625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45851 (local address 127.10.236.254:34381)
0926 13:49:58.607788 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.607794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.608773 (+ 979us) server_negotiation.cc:407] Connection header received
0926 13:49:58.609964 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.609970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.610038 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.610106 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:58.611629 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.612210 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.613356 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.613574 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.613661 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.616195 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:58.616199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:58.616201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:58.616228 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:58.617762 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.618330 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.618332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.618334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.618413 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.618826 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.618828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.618829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.619015 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:58.619089 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.619747 (+ 658us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.619857 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 13:49:58.620874 12218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.608115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34381 (local address 127.10.236.195:45851)
0926 13:49:58.608610 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.608654 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.609719 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.610304 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.610331 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.610847 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:49:58.611369 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.611407 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.612326 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.612329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.613222 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.613231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.613724 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.613727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.613871 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.614415 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:49:58.614431 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:49:58.615987 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:49:58.617905 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:58.617911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:58.617923 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:58.618208 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:58.618493 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:49:58.618495 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:49:58.618497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:49:58.618638 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:49:58.619118 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:49:58.619123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:49:58.619436 (+ 313us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.619639 (+ 203us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.619988 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
I20260926 13:49:58.622247 12216 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34381
I20260926 13:49:58.622566 12216 heartbeater.cc:464] Registering TS with master...
I20260926 13:49:58.623212 12216 heartbeater.cc:511] Master 127.10.236.254:34381 requested a full tablet report, sending...
I20260926 13:49:58.624544 11757 ts_manager.cc:194] Registered new tserver with Master: 84491570eb544dafbdaf4aad89cac87e (127.10.236.195:41769)
I20260926 13:49:58.625177 11757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:45851
I20260926 13:49:58.635831 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:49:58.647631 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.639623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.10.236.254:34381)
0926 13:49:58.639790 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.639795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.639991 (+ 196us) server_negotiation.cc:407] Connection header received
0926 13:49:58.640127 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.640130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.640184 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.640271 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:58.641109 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.641764 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.642709 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.642943 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.643047 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.644562 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:58.644565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:58.644568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:58.644595 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:58.645914 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.646341 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.646346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.646347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.646429 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.646704 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.646708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.646709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.646934 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:58.647077 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.647315 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.647426 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:49:58.652585 11757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:49:58.655627 11757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:49:58.670576 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.664642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:41769 (local address 127.0.0.1:43240)
0926 13:49:58.665608 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.665622 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.665787 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.666445 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.666449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.666470 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:49:58.666577 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.666582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.667646 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.667650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.669087 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.669104 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.670034 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.670037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.670138 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.670172 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.670291 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.670410 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":96,"threads_started":1}
I20260926 13:49:58.670591 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.665017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.10.236.195:41769)
0926 13:49:58.665442 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.665457 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.665713 (+ 256us) server_negotiation.cc:407] Connection header received
0926 13:49:58.665951 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.665959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.666157 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.666273 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:49:58.666673 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.667541 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.669223 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.669910 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.670049 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.670187 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.670316 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.670417 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":94,"threads_started":1}
I20260926 13:49:58.671190 12230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.665374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:33691 (local address 127.0.0.1:33086)
0926 13:49:58.666006 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.666019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.666112 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.666501 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.666504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.666522 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:49:58.666633 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.666641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.668284 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.668287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.669423 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.669433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.670784 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.670788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.670882 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.670906 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.670984 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.671050 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":77,"threads_started":1}
I20260926 13:49:58.671643 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.665523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.10.236.193:33691)
0926 13:49:58.665901 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.665910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.666096 (+ 186us) server_negotiation.cc:407] Connection header received
0926 13:49:58.666225 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.666237 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.666378 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.666518 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:49:58.666771 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.668049 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.669557 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.670448 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.670575 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.670721 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.671365 (+ 644us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.671463 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":119,"threads_started":1}
I20260926 13:49:58.673441 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.664625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:35521 (local address 127.0.0.1:46634)
0926 13:49:58.665003 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.665049 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.665167 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.667415 (+ 2248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.667418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.667437 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:49:58.667545 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.667551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.670567 (+ 3016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.670570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.671824 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.671834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.673055 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.673059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.673148 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.673171 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.673238 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.673313 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":162,"threads_started":1}
I20260926 13:49:58.673502 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.665657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.10.236.194:35521)
0926 13:49:58.666870 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.666887 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.666977 (+ 90us) server_negotiation.cc:407] Connection header received
0926 13:49:58.667057 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.667062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.667260 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.667409 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:49:58.669232 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.670390 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.671986 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.672873 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.673069 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.673210 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.673277 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.673361 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":115,"threads_started":1}
I20260926 13:49:58.674032 12150 tablet_service.cc:1467] Processing CreateTablet for tablet 7d79e692684640efab0c391639a08572 (DEFAULT_TABLE table=test-table [id=ed42001abb67482ba331532e4ca018f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:49:58.674315 11882 tablet_service.cc:1467] Processing CreateTablet for tablet 7d79e692684640efab0c391639a08572 (DEFAULT_TABLE table=test-table [id=ed42001abb67482ba331532e4ca018f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:49:58.675184 11882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d79e692684640efab0c391639a08572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:49:58.675189 12150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d79e692684640efab0c391639a08572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:49:58.676528 12016 tablet_service.cc:1467] Processing CreateTablet for tablet 7d79e692684640efab0c391639a08572 (DEFAULT_TABLE table=test-table [id=ed42001abb67482ba331532e4ca018f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:49:58.677575 12016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d79e692684640efab0c391639a08572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:49:58.682463 12233 tablet_bootstrap.cc:492] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: Bootstrap starting.
I20260926 13:49:58.683162 12234 tablet_bootstrap.cc:492] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: Bootstrap starting.
I20260926 13:49:58.685110 12233 tablet_bootstrap.cc:654] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: Neither blocks nor log segments found. Creating new log.
I20260926 13:49:58.685101 12235 tablet_bootstrap.cc:492] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: Bootstrap starting.
I20260926 13:49:58.685273 12234 tablet_bootstrap.cc:654] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: Neither blocks nor log segments found. Creating new log.
I20260926 13:49:58.686036 12233 log.cc:824] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:49:58.686475 12234 log.cc:824] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:49:58.688177 12235 tablet_bootstrap.cc:654] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: Neither blocks nor log segments found. Creating new log.
I20260926 13:49:58.688748 12233 tablet_bootstrap.cc:492] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: No bootstrap required, opened a new log
I20260926 13:49:58.688865 12235 log.cc:824] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: Log is configured to *not* fsync() on all Append() calls
I20260926 13:49:58.688977 12233 ts_tablet_manager.cc:1397] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260926 13:49:58.689291 12234 tablet_bootstrap.cc:492] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: No bootstrap required, opened a new log
I20260926 13:49:58.689484 12234 ts_tablet_manager.cc:1397] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:49:58.690523 12235 tablet_bootstrap.cc:492] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: No bootstrap required, opened a new log
I20260926 13:49:58.690712 12235 ts_tablet_manager.cc:1397] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:49:58.693523 12235 raft_consensus.cc:348] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.693819 12235 raft_consensus.cc:374] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:49:58.693874 12235 raft_consensus.cc:729] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011626ecbb3e4c8a9c71d183e97dea77, State: Initialized, Role: FOLLOWER
I20260926 13:49:58.693780 12234 raft_consensus.cc:348] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.693856 12233 raft_consensus.cc:348] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.694054 12234 raft_consensus.cc:374] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:49:58.694054 12233 raft_consensus.cc:374] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:49:58.694105 12234 raft_consensus.cc:729] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84491570eb544dafbdaf4aad89cac87e, State: Initialized, Role: FOLLOWER
I20260926 13:49:58.694113 12233 raft_consensus.cc:729] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f75b7f735144f9387d4f4eb1f9c892d, State: Initialized, Role: FOLLOWER
I20260926 13:49:58.694388 12235 consensus_queue.cc:260] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.694531 12233 consensus_queue.cc:260] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.694531 12234 consensus_queue.cc:260] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.695401 12235 ts_tablet_manager.cc:1428] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 13:49:58.695457 12233 ts_tablet_manager.cc:1428] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:49:58.695457 12234 ts_tablet_manager.cc:1428] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:49:58.695829 12216 heartbeater.cc:503] Master 127.10.236.254:34381 was elected leader, sending a full tablet report...
I20260926 13:49:58.695955 12082 heartbeater.cc:503] Master 127.10.236.254:34381 was elected leader, sending a full tablet report...
I20260926 13:49:58.695955 11948 heartbeater.cc:503] Master 127.10.236.254:34381 was elected leader, sending a full tablet report...
W20260926 13:49:58.856819 12217 tablet.cc:2369] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:49:58.874671 12083 tablet.cc:2369] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:49:58.893877 11949 tablet.cc:2369] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:49:58.918270 12241 raft_consensus.cc:482] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:49:58.918573 12241 raft_consensus.cc:504] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.920082 12241 leader_election.cc:288] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011626ecbb3e4c8a9c71d183e97dea77 (127.10.236.194:35521), 9f75b7f735144f9387d4f4eb1f9c892d (127.10.236.193:33691)
I20260926 13:49:58.924634 12218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.920430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:35521 (local address 127.10.236.195:45511)
0926 13:49:58.920566 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.920585 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.920684 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.921047 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.921052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.921075 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:49:58.921196 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.921206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.922175 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.922179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.923144 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.923153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.924120 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.924124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.924253 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.924280 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.924369 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.924444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 13:49:58.924638 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.920569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45511 (local address 127.10.236.194:35521)
0926 13:49:58.920758 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.920764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.920783 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:49:58.920856 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.920858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.920931 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.921031 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:49:58.921328 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.922043 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.923286 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.923996 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.924104 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.924149 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.924435 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.924502 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 13:49:58.925212 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.920721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:33691 (local address 127.10.236.195:55527)
0926 13:49:58.921114 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0926 13:49:58.921130 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:49:58.921233 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:49:58.921506 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:49:58.921509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:49:58.921529 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:49:58.921669 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.921676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.922708 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.922711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.923787 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:49:58.923798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.924775 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.924780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:49:58.924902 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.924929 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:49:58.925016 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 13:49:58.925085 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":156,"threads_started":1}
I20260926 13:49:58.925228 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.920912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:55527 (local address 127.10.236.193:33691)
0926 13:49:58.921042 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.921046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.921220 (+ 174us) server_negotiation.cc:407] Connection header received
0926 13:49:58.921366 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.921368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.921409 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.921464 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:49:58.921854 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.922551 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.923929 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.924576 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.924706 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.924814 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.925031 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.925126 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:49:58.925443 12036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79e692684640efab0c391639a08572" candidate_uuid: "84491570eb544dafbdaf4aad89cac87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011626ecbb3e4c8a9c71d183e97dea77" is_pre_election: true
I20260926 13:49:58.925813 12036 raft_consensus.cc:2454] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84491570eb544dafbdaf4aad89cac87e in term 0.
I20260926 13:49:58.925987 11902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79e692684640efab0c391639a08572" candidate_uuid: "84491570eb544dafbdaf4aad89cac87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" is_pre_election: true
I20260926 13:49:58.926275 11902 raft_consensus.cc:2454] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84491570eb544dafbdaf4aad89cac87e in term 0.
I20260926 13:49:58.926338 12106 leader_election.cc:302] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011626ecbb3e4c8a9c71d183e97dea77, 84491570eb544dafbdaf4aad89cac87e; no voters:
I20260926 13:49:58.926623 12241 raft_consensus.cc:2781] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:49:58.926713 12241 raft_consensus.cc:482] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:49:58.926755 12241 raft_consensus.cc:3037] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:49:58.927845 12241 raft_consensus.cc:504] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.928184 12241 leader_election.cc:288] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [CANDIDATE]: Term 1 election: Requested vote from peers 011626ecbb3e4c8a9c71d183e97dea77 (127.10.236.194:35521), 9f75b7f735144f9387d4f4eb1f9c892d (127.10.236.193:33691)
I20260926 13:49:58.928610 12036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79e692684640efab0c391639a08572" candidate_uuid: "84491570eb544dafbdaf4aad89cac87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011626ecbb3e4c8a9c71d183e97dea77"
I20260926 13:49:58.928606 11902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79e692684640efab0c391639a08572" candidate_uuid: "84491570eb544dafbdaf4aad89cac87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f75b7f735144f9387d4f4eb1f9c892d"
I20260926 13:49:58.928764 11902 raft_consensus.cc:3037] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:49:58.928751 12036 raft_consensus.cc:3037] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:49:58.930461 11902 raft_consensus.cc:2454] T 7d79e692684640efab0c391639a08572 P 9f75b7f735144f9387d4f4eb1f9c892d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84491570eb544dafbdaf4aad89cac87e in term 1.
I20260926 13:49:58.930461 12036 raft_consensus.cc:2454] T 7d79e692684640efab0c391639a08572 P 011626ecbb3e4c8a9c71d183e97dea77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84491570eb544dafbdaf4aad89cac87e in term 1.
I20260926 13:49:58.930797 12106 leader_election.cc:302] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84491570eb544dafbdaf4aad89cac87e, 9f75b7f735144f9387d4f4eb1f9c892d; no voters:
I20260926 13:49:58.930985 12241 raft_consensus.cc:2781] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:49:58.931213 12241 raft_consensus.cc:686] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [term 1 LEADER]: Becoming Leader. State: Replica: 84491570eb544dafbdaf4aad89cac87e, State: Running, Role: LEADER
I20260926 13:49:58.931625 12241 consensus_queue.cc:237] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } }
I20260926 13:49:58.934995 11755 catalog_manager.cc:5495] T 7d79e692684640efab0c391639a08572 P 84491570eb544dafbdaf4aad89cac87e reported cstate change: term changed from 0 to 1, leader changed from <none> to 84491570eb544dafbdaf4aad89cac87e (127.10.236.195). New cstate: current_term: 1 leader_uuid: "84491570eb544dafbdaf4aad89cac87e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011626ecbb3e4c8a9c71d183e97dea77" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84491570eb544dafbdaf4aad89cac87e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f75b7f735144f9387d4f4eb1f9c892d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33691 } health_report { overall_health: UNKNOWN } } }
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:49:58.967566 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.958083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.10.236.193:33691)
0926 13:49:58.958264 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.958268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.958283 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:49:58.958455 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.958457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.958512 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.958593 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:58.959383 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.959996 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.960864 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.961064 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.961140 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.963945 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:58.963959 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:58.963975 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:58.964009 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:58.965727 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.966227 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.966233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.966235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.966319 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.966607 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.966610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.966612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.966869 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:58.967093 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.967250 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.967374 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:49:58 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:49:58.997090 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:58.987596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.10.236.193:33691)
0926 13:49:58.987790 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:58.987794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:58.988004 (+ 210us) server_negotiation.cc:407] Connection header received
0926 13:49:58.988193 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:58.988196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:58.988249 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:58.988336 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:58.989031 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.989549 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.990393 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:58.990621 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:58.990727 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:58.993295 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:58.993299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:58.993301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:58.993327 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:58.995080 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.995596 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.995601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.995602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.995691 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:58.996040 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:58.996043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:58.996045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:58.996226 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:58.996363 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:58.996749 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 13:49:58.996906 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Sep 26 13:49:59 dist-test-slave-r34j krb5kdc[11712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:49:59 dist-test-slave-r34j krb5kdc[11712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:49:59.024684 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:49:59.015570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.10.236.193:33691)
0926 13:49:59.015719 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:49:59.015725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:49:59.015853 (+ 128us) server_negotiation.cc:407] Connection header received
0926 13:49:59.015971 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:49:59.015973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:49:59.016017 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:49:59.016096 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:49:59.016833 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:59.017343 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:59.018246 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:49:59.018477 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:49:59.018568 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:49:59.021386 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:49:59.021390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:49:59.021392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:49:59.021417 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:49:59.022972 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:59.023403 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:59.023408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:59.023410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:59.023530 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:49:59.023832 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:49:59.023837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:49:59.023839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:49:59.024006 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:49:59.024141 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 13:49:59.024388 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 13:49:59.024511 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 13:49:59.026055 11862 tablet_service.cc:2908] Rejecting scan request for tablet 7d79e692684640efab0c391639a08572: Uninitialized: clean time has not yet been initialized
I20260926 13:49:59.028247 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11819
I20260926 13:49:59.034116 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11953
I20260926 13:49:59.039831 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 12087
I20260926 13:49:59.045248 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 11725
2026-09-26T13:49:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2544 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:49:59 dist-test-slave-r34j krb5kdc[12257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:49:59 dist-test-slave-r34j krb5kdc[12257](info): set up 2 sockets
Sep 26 13:49:59 dist-test-slave-r34j krb5kdc[12257](info): commencing operation
krb5kdc: starting...
W20260926 13:50:00.091334 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:00.222563 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44197
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:00.338954 12270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:00.339279 12270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:00.339418 12270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:00.343760 12270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:00.343918 12270 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 13:50:00.343953 12270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:00.343995 12270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:00.344033 12270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:00.344069 12270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:00.349838 12270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44197
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:00.351013 12270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:00.352133 12270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:00.360965 12276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:00.360953 12278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:00.361222 12270 server_base.cc:1034] running on GCE node
W20260926 13:50:00.361030 12275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:00.361935 12270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:00.363041 12270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:00.364229 12270 hybrid_clock.cc:648] HybridClock initialized: now 1790430600364179 us; error 65 us; skew 500 ppm
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:00.377635 12270 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:00.378737 12270 webserver.cc:460] Webserver started at http://127.10.236.254:37103/ using document root <none> and password file <none>
I20260926 13:50:00.379401 12270 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:00.379532 12270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:00.379779 12270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:00.381942 12270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "f8e7b66c93ba48afbaaf1c90181706a7"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "55cc235824e3089061ba8f4a3dfa994d"
server_key_iv: "a0f0fcdeaa20271d8e4a48eb5c3e9bbd"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.382536 12270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "f8e7b66c93ba48afbaaf1c90181706a7"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "55cc235824e3089061ba8f4a3dfa994d"
server_key_iv: "a0f0fcdeaa20271d8e4a48eb5c3e9bbd"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.386518 12270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260926 13:50:00.389235 12285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:00.390635 12270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:50:00.390830 12270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "f8e7b66c93ba48afbaaf1c90181706a7"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "55cc235824e3089061ba8f4a3dfa994d"
server_key_iv: "a0f0fcdeaa20271d8e4a48eb5c3e9bbd"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.390960 12270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:00.413336 12270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:00.417289 12270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:00.417528 12270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:00.426100 12270 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44197
I20260926 13:50:00.426169 12347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44197 every 8 connection(s)
I20260926 13:50:00.427194 12270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:00.430112 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 12270
I20260926 13:50:00.430238 12348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:00.430298 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:00.430598 11187 external_mini_cluster.cc:1428] Setting key 7fe609720ec922ba4b90a56017d0b367
I20260926 13:50:00.438567 12348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: Bootstrap starting.
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:00.441453 12348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:00.442196 12348 log.cc:824] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:00.444434 12348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: No bootstrap required, opened a new log
I20260926 13:50:00.447232 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.432190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.10.236.254:44197)
0926 13:50:00.432643 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:00.432662 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:00.432707 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:50:00.433408 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:00.433425 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:00.433795 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:00.434097 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:00.435088 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.436216 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.437460 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.437970 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.438094 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.441075 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:00.441081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:00.441095 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:00.441126 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:00.443720 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.444201 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.444212 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.444216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.444340 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.444675 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.444679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.444681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.445225 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:00.445432 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:00.445742 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 13:50:00.446086 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":130,"threads_started":1}
I20260926 13:50:00.448760 12348 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44197 } }
I20260926 13:50:00.449043 12348 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:00.449095 12348 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e7b66c93ba48afbaaf1c90181706a7, State: Initialized, Role: FOLLOWER
I20260926 13:50:00.449558 12348 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44197 } }
I20260926 13:50:00.449718 12348 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:00.449791 12348 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:00.449889 12348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:00.450881 12348 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44197 } }
I20260926 13:50:00.451241 12348 leader_election.cc:302] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e7b66c93ba48afbaaf1c90181706a7; no voters:
I20260926 13:50:00.451642 12348 leader_election.cc:288] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:00.451880 12353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:00.452153 12353 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [term 1 LEADER]: Becoming Leader. State: Replica: f8e7b66c93ba48afbaaf1c90181706a7, State: Running, Role: LEADER
I20260926 13:50:00.452486 12353 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44197 } }
I20260926 13:50:00.453023 12348 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:00.454315 12355 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e7b66c93ba48afbaaf1c90181706a7. Latest consensus state: current_term: 1 leader_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44197 } } }
I20260926 13:50:00.454452 12355 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:00.454792 12354 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e7b66c93ba48afbaaf1c90181706a7" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44197 } } }
I20260926 13:50:00.454867 12362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:00.455526 12354 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:00.457794 12362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:00.463707 12362 catalog_manager.cc:1347] Generated new cluster ID: 3248c46c449d493dbe214dc58b36a9fd
I20260926 13:50:00.463792 12362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:00.496918 12362 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:00.497978 12362 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:00.504879 12362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: Generated new TSK 0
I20260926 13:50:00.505817 12362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:00.538995 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44197
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:00.663300 12374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:00.663642 12374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:00.663689 12374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:00.668062 12374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:00.668129 12374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:00.668170 12374 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:00.668202 12374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:00.673593 12374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:00.674575 12374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:00.675565 12374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:00.685176 12379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:00.685220 12382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:00.685277 12374 server_base.cc:1034] running on GCE node
W20260926 13:50:00.685220 12380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:00.685817 12374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:00.686480 12374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:00.687675 12374 hybrid_clock.cc:648] HybridClock initialized: now 1790430600687641 us; error 52 us; skew 500 ppm
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:00.690416 12374 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:00.691275 12374 webserver.cc:460] Webserver started at http://127.10.236.193:39075/ using document root <none> and password file <none>
I20260926 13:50:00.691900 12374 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:00.691991 12374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:00.692225 12374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:00.694326 12374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "241a317b24b54e3998faa5afd6685643"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "a9f47c277bd52384c5c36694630bacf0"
server_key_iv: "d5059e55bad05a9979b6974f6af16cf9"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.694902 12374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "241a317b24b54e3998faa5afd6685643"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "a9f47c277bd52384c5c36694630bacf0"
server_key_iv: "d5059e55bad05a9979b6974f6af16cf9"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.698757 12374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:00.701306 12389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:00.702651 12374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:00.702800 12374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "241a317b24b54e3998faa5afd6685643"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "a9f47c277bd52384c5c36694630bacf0"
server_key_iv: "d5059e55bad05a9979b6974f6af16cf9"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.702915 12374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:00.712793 12374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:00.716018 12374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:00.716210 12374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:00.717602 12374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:00.717650 12374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:00.717692 12374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:00.717707 12374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:00.728756 12374 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:45161
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:00.728943 12504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:45161 every 8 connection(s)
I20260926 13:50:00.729705 12374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:00.733357 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.718651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:35161 (local address 127.10.236.254:44197)
0926 13:50:00.718789 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:00.718793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:00.719868 (+ 1075us) server_negotiation.cc:407] Connection header received
0926 13:50:00.721133 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:00.721136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:00.721196 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:00.721302 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:00.722944 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.723866 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.725766 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.725994 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.726078 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.729500 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:00.729504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:00.729507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:00.729536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:00.731224 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.731819 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.731824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.731825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.731914 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.732257 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.732260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.732262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.732454 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:00.732558 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:00.733053 (+ 495us) server_negotiation.cc:299] Negotiation successful
0926 13:50:00.733188 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:00.734107 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.719164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.10.236.193:35161)
0926 13:50:00.719715 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:00.719762 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:00.720887 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:00.721504 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:00.721515 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:00.721945 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:00.722749 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.722771 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.724045 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.724051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.725629 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.725641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.726169 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.726174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.726398 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.727177 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:00.727203 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:00.729306 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:00.731348 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.731354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.731369 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.731714 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.732014 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.732017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.732019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.732156 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.732568 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:00.732576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:00.732820 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 13:50:00.733000 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 13:50:00.733313 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":131,"threads_started":1}
I20260926 13:50:00.735554 12505 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44197
I20260926 13:50:00.735741 12505 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:00.736346 12505 heartbeater.cc:511] Master 127.10.236.254:44197 requested a full tablet report, sending...
I20260926 13:50:00.737684 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 12374
I20260926 13:50:00.737808 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:00.738060 11187 external_mini_cluster.cc:1428] Setting key 83de560d51ff09aeefe94cbe492186da
I20260926 13:50:00.738106 12302 ts_manager.cc:194] Registered new tserver with Master: 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161)
I20260926 13:50:00.739905 12302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:35161
I20260926 13:50:00.746632 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.737799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:42201 (local address 127.10.236.254:44197)
0926 13:50:00.738405 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:00.738408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:00.739185 (+ 777us) server_negotiation.cc:407] Connection header received
0926 13:50:00.739291 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:00.739295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:00.739338 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:00.739416 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:00.739952 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.740578 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.741767 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.742144 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.742277 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.743587 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:00.743590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:00.743592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:00.743613 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:00.745026 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.745458 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.745463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.745465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.745542 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.745823 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.745826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.745828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.745996 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:00.746104 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:00.746360 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 13:50:00.746469 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535}
I20260926 13:50:00.746655 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.737704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.10.236.193:42201)
0926 13:50:00.739065 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:00.739082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:00.739198 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:00.739466 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:00.739469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:00.739719 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:00.739825 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.739831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.740732 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.740736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.741626 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.741638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.742302 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.742306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.742415 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.743024 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:00.743036 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:00.743392 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:00.745141 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.745147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.745150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.745359 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.745633 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.745637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.745639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.745728 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.746092 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:00.746095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:00.746194 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 13:50:00.746309 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 13:50:00.746457 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1268,"mutex_wait_us":1,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:00.767449 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44197
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:00.881099 12514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:00.881394 12514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:00.881465 12514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:00.885706 12514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:00.885802 12514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:00.885881 12514 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:00.885960 12514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:00.891252 12514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:00.892386 12514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:00.893527 12514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:00.903451 12522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:00.903514 12519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:00.903446 12520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:00.903864 12514 server_base.cc:1034] running on GCE node
I20260926 13:50:00.904295 12514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:00.904963 12514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:00.906152 12514 hybrid_clock.cc:648] HybridClock initialized: now 1790430600906131 us; error 33 us; skew 500 ppm
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:00.909111 12514 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:00.910013 12514 webserver.cc:460] Webserver started at http://127.10.236.194:46553/ using document root <none> and password file <none>
I20260926 13:50:00.910578 12514 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:00.910638 12514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:00.910851 12514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:00.912971 12514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "b7fc887bee4b4c1286dc7408c06edc34"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "a555a5ef59adc4cc311392c84ebb5391"
server_key_iv: "0384318887ce3121b65b431b2bedc15d"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.913534 12514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "b7fc887bee4b4c1286dc7408c06edc34"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "a555a5ef59adc4cc311392c84ebb5391"
server_key_iv: "0384318887ce3121b65b431b2bedc15d"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.917191 12514 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:50:00.919651 12529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:00.920938 12514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:00.921069 12514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "b7fc887bee4b4c1286dc7408c06edc34"
format_stamp: "Formatted at 2026-09-26 13:50:00 on dist-test-slave-r34j"
server_key: "a555a5ef59adc4cc311392c84ebb5391"
server_key_iv: "0384318887ce3121b65b431b2bedc15d"
server_key_version: "encryptionkey@0"
I20260926 13:50:00.921182 12514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:00.933822 12514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:00.937198 12514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:00.937428 12514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:00.938841 12514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:00.938912 12514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:00.938978 12514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:00.939020 12514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:50:00 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:00.949983 12514 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:44077
I20260926 13:50:00.950053 12644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:44077 every 8 connection(s)
I20260926 13:50:00.951171 12514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:00.953898 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.939983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:37871 (local address 127.10.236.254:44197)
0926 13:50:00.940118 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:00.940122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:00.941202 (+ 1080us) server_negotiation.cc:407] Connection header received
0926 13:50:00.942307 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:00.942311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:00.942370 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:00.942468 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:00.944160 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.944770 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.946662 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.946962 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.947050 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.950070 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:00.950074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:00.950077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:00.950105 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:00.951742 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.952347 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.952351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.952353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.952438 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.952806 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.952810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.952811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.953000 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:00.953105 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:00.953614 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 13:50:00.953732 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:50:00.954547 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 12514
I20260926 13:50:00.954681 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:00.954694 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.940479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.10.236.194:37871)
0926 13:50:00.941035 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:00.941084 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:00.942094 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:00.942691 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:00.942703 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:00.943167 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:00.943965 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.943991 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.944879 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.944885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.946500 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.946514 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.947119 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.947123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.947363 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.948194 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:00.948218 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:00.949882 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:00.951867 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.951872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.951887 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.952231 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.952554 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.952558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.952560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.952703 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.953102 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:00.953108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:00.953352 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 13:50:00.953564 (+ 212us) client_negotiation.cc:240] Negotiation successful
0926 13:50:00.953881 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":140,"threads_started":1}
I20260926 13:50:00.954984 11187 external_mini_cluster.cc:1428] Setting key 8f7f8fc57387eee61b39b8e2649179bb
I20260926 13:50:00.958017 12645 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44197
I20260926 13:50:00.958171 12645 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:00.958683 12645 heartbeater.cc:511] Master 127.10.236.254:44197 requested a full tablet report, sending...
I20260926 13:50:00.959803 12301 ts_manager.cc:194] Registered new tserver with Master: b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077)
I20260926 13:50:00.960402 12301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:37871
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:00.966678 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.958403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:55323 (local address 127.10.236.254:44197)
0926 13:50:00.958553 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:00.958558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:00.959521 (+ 963us) server_negotiation.cc:407] Connection header received
0926 13:50:00.959693 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:00.959696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:00.959755 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:00.959841 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:00.960279 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.960962 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.961877 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.962108 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.962217 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.963437 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:00.963441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:00.963444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:00.963469 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:00.964979 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.965429 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.965433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.965435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.965520 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:00.965805 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:00.965808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:00.965810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:00.966025 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:00.966136 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:00.966402 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 13:50:00.966514 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 13:50:00.966687 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:00.958362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.10.236.194:55323)
0926 13:50:00.959408 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:00.959422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:00.959581 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:00.959856 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:00.959859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:00.960065 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:00.960159 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.960164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.961069 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.961073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.961761 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:00.961770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:00.962220 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:00.962224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:00.962372 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:00.962931 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:00.962944 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:00.963285 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:00.965111 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.965115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.965117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.965328 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.965617 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:00.965620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:00.965622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:00.965708 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:00.966124 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:00.966127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:00.966226 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 13:50:00.966338 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 13:50:00.966486 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:00.987097 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44197
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:01.100353 12654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:01.100630 12654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:01.100706 12654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:01.104943 12654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:01.105029 12654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:01.105096 12654 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:01.105181 12654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:01.110723 12654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:01.111902 12654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:01.112959 12654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:01.122661 12660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:01.122685 12659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:01.122915 12662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:01.123157 12654 server_base.cc:1034] running on GCE node
I20260926 13:50:01.123631 12654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:01.124214 12654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:01.125432 12654 hybrid_clock.cc:648] HybridClock initialized: now 1790430601125403 us; error 35 us; skew 500 ppm
Sep 26 13:50:01 dist-test-slave-r34j krb5kdc[12257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:01.128371 12654 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:01.129331 12654 webserver.cc:460] Webserver started at http://127.10.236.195:32771/ using document root <none> and password file <none>
I20260926 13:50:01.129908 12654 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:01.130002 12654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:01.130234 12654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:01.132444 12654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "ea449e70c02649938a2618cf41add202"
format_stamp: "Formatted at 2026-09-26 13:50:01 on dist-test-slave-r34j"
server_key: "e382aaa8b20966e0fe745b307494cc33"
server_key_iv: "3161e8e0b9eaf9e29366bb5900f1f4cf"
server_key_version: "encryptionkey@0"
I20260926 13:50:01.133033 12654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "ea449e70c02649938a2618cf41add202"
format_stamp: "Formatted at 2026-09-26 13:50:01 on dist-test-slave-r34j"
server_key: "e382aaa8b20966e0fe745b307494cc33"
server_key_iv: "3161e8e0b9eaf9e29366bb5900f1f4cf"
server_key_version: "encryptionkey@0"
I20260926 13:50:01.137113 12654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:01.139819 12669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:01.141162 12654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:01.141337 12654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "ea449e70c02649938a2618cf41add202"
format_stamp: "Formatted at 2026-09-26 13:50:01 on dist-test-slave-r34j"
server_key: "e382aaa8b20966e0fe745b307494cc33"
server_key_iv: "3161e8e0b9eaf9e29366bb5900f1f4cf"
server_key_version: "encryptionkey@0"
I20260926 13:50:01.141467 12654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:01.171152 12654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:01.174744 12654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:01.174976 12654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:01.176578 12654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:01.176677 12654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:01.176760 12654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:01.176800 12654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:50:01 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430601, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:01.189213 12654 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:35179
I20260926 13:50:01.189339 12784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:35179 every 8 connection(s)
I20260926 13:50:01.190483 12654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:01.192129 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.177465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:37119 (local address 127.10.236.254:44197)
0926 13:50:01.177598 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.177602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.178765 (+ 1163us) server_negotiation.cc:407] Connection header received
0926 13:50:01.179924 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.179927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.179985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.180042 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:01.181834 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.182461 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.184062 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.184334 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.184453 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.187732 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:01.187736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:01.187739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:01.187772 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:01.189632 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.190189 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.190193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.190195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.190297 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.190759 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.190764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.190766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.191047 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:01.191156 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.191906 (+ 750us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.192013 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:50:01.192981 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.178054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.10.236.195:37119)
0926 13:50:01.178576 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.178618 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.179659 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.180535 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.180544 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.181057 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:01.181678 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.181699 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.182580 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.182584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.183937 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.183946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.184511 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.184515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.184751 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.185524 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:01.185551 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:01.187535 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:01.189752 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:01.189757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:01.189773 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:01.190049 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:01.190422 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:01.190425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:01.190427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:01.190645 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:01.191254 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:01.191262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:01.191613 (+ 351us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.191784 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.192045 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":146,"threads_started":1}
I20260926 13:50:01.194152 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 12654
I20260926 13:50:01.194293 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:01.194379 12785 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44197
I20260926 13:50:01.194545 12785 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:01.194633 11187 external_mini_cluster.cc:1428] Setting key c9a8808298234ccad45e711a5ebee619
I20260926 13:50:01.195154 12785 heartbeater.cc:511] Master 127.10.236.254:44197 requested a full tablet report, sending...
I20260926 13:50:01.196274 12301 ts_manager.cc:194] Registered new tserver with Master: ea449e70c02649938a2618cf41add202 (127.10.236.195:35179)
I20260926 13:50:01.197048 12301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:37119
I20260926 13:50:01.204581 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.196759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.10.236.195:55129)
0926 13:50:01.197441 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.197456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.197553 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.197766 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.197768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.197978 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:01.198075 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.198081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.199175 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.199178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.199889 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.199899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.200353 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.200356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.200428 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.201018 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:01.201039 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:01.201368 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:01.203005 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:01.203007 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:01.203009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:01.203200 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:01.203544 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:01.203546 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:01.203548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:01.203632 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:01.204092 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:01.204095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:01.204192 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.204300 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.204428 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":89,"threads_started":1}
I20260926 13:50:01.204705 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.196808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:55129 (local address 127.10.236.254:44197)
0926 13:50:01.196924 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.196928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.197534 (+ 606us) server_negotiation.cc:407] Connection header received
0926 13:50:01.197641 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.197644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.197680 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.197736 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:01.198192 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.199031 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.200012 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.200232 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.200317 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.201501 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:01.201504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:01.201506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:01.201528 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:01.202902 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.203297 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.203300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.203302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.203397 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.203741 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.203744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.203745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.203948 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:01.204044 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.204405 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.204506 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:50:01.209167 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:50:01 dist-test-slave-r34j krb5kdc[12257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:50:01 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:01.236007 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.227124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.10.236.254:44197)
0926 13:50:01.227265 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.227269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.227283 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:01.227328 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.227330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.227371 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.227442 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:01.228246 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.228777 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.229714 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.229936 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.230043 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.232617 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:01.232620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:01.232631 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:01.232659 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:01.234308 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.234699 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.234702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.234703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.234785 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.235112 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.235115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.235116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.235340 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:01.235462 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.235772 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.235886 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:50:01.239291 12301 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:01.242363 12301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:01.256740 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.250906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.10.236.193:45161)
0926 13:50:01.251270 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.251277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.251301 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:50:01.251531 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.251537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.251760 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.251923 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.252454 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.253498 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.255114 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.256070 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.256205 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.256397 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.256499 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.256592 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":84,"threads_started":1}
I20260926 13:50:01.256742 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.250709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:45161 (local address 127.0.0.1:43350)
0926 13:50:01.251117 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.251186 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.251305 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.252102 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.252108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.252134 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.252325 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.252332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.253611 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.253614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.254993 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.255002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.256191 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.256194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.256311 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.256348 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.256470 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.256592 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"spinlock_wait_cycles":2816,"thread_start_us":132,"threads_started":1}
I20260926 13:50:01.257400 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.251070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35179 (local address 127.0.0.1:40118)
0926 13:50:01.252109 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.252121 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.252205 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.252915 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.252918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.252936 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.253047 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.253053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.254421 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.254424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.255612 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.255621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.256983 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.256986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.257068 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.257088 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.257175 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.257244 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":52,"spinlock_wait_cycles":2816,"thread_start_us":85,"threads_started":1}
I20260926 13:50:01.257447 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.251570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.10.236.195:35179)
0926 13:50:01.252444 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.252463 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.252483 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:50:01.252574 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.252580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.252771 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.252922 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.253165 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.254273 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.255728 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.256618 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.256761 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.256919 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.257216 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.257312 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":133,"threads_started":1}
I20260926 13:50:01.258189 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.251917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.10.236.194:44077)
0926 13:50:01.252292 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.252300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.252548 (+ 248us) server_negotiation.cc:407] Connection header received
0926 13:50:01.252644 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.252649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.252791 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.252922 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.253331 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.254177 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.255880 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.256629 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.256761 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.257932 (+ 1171us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.257980 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.258059 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260926 13:50:01.259804 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.251513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44077 (local address 127.0.0.1:39354)
0926 13:50:01.252435 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.252448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.252547 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.253062 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.253064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.253079 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.253207 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.253216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.254312 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.254318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.255757 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.255768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.257171 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.257174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.257260 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.257280 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.257335 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.257387 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":228,"thread_start_us":82,"threads_started":1}
I20260926 13:50:01.260092 12719 tablet_service.cc:1467] Processing CreateTablet for tablet 09a8fdf843da46b3bf3b960b1f8ac512 (DEFAULT_TABLE table=test-table [id=f9fba79496b34f87b1d0451e8855de5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:01.260089 12439 tablet_service.cc:1467] Processing CreateTablet for tablet 09a8fdf843da46b3bf3b960b1f8ac512 (DEFAULT_TABLE table=test-table [id=f9fba79496b34f87b1d0451e8855de5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:01.261256 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a8fdf843da46b3bf3b960b1f8ac512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.261260 12719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a8fdf843da46b3bf3b960b1f8ac512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.262555 12579 tablet_service.cc:1467] Processing CreateTablet for tablet 09a8fdf843da46b3bf3b960b1f8ac512 (DEFAULT_TABLE table=test-table [id=f9fba79496b34f87b1d0451e8855de5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:01.263348 12579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a8fdf843da46b3bf3b960b1f8ac512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.269528 12810 tablet_bootstrap.cc:492] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Bootstrap starting.
I20260926 13:50:01.269982 12808 tablet_bootstrap.cc:492] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Bootstrap starting.
I20260926 13:50:01.271634 12810 tablet_bootstrap.cc:654] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.271749 12808 tablet_bootstrap.cc:654] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.271827 12809 tablet_bootstrap.cc:492] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Bootstrap starting.
I20260926 13:50:01.272425 12810 log.cc:824] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:01.272555 12808 log.cc:824] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:01.273931 12809 tablet_bootstrap.cc:654] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.274432 12808 tablet_bootstrap.cc:492] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: No bootstrap required, opened a new log
I20260926 13:50:01.274521 12810 tablet_bootstrap.cc:492] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: No bootstrap required, opened a new log
I20260926 13:50:01.274624 12808 ts_tablet_manager.cc:1397] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:50:01.274682 12810 ts_tablet_manager.cc:1397] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260926 13:50:01.274703 12809 log.cc:824] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:01.276716 12809 tablet_bootstrap.cc:492] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: No bootstrap required, opened a new log
I20260926 13:50:01.276901 12809 ts_tablet_manager.cc:1397] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 13:50:01.278105 12808 raft_consensus.cc:348] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.278316 12808 raft_consensus.cc:374] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.278347 12808 raft_consensus.cc:729] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 241a317b24b54e3998faa5afd6685643, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.278496 12810 raft_consensus.cc:348] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.278843 12810 raft_consensus.cc:374] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.278786 12808 consensus_queue.cc:260] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.278916 12810 raft_consensus.cc:729] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc887bee4b4c1286dc7408c06edc34, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.279314 12810 consensus_queue.cc:260] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.279454 12505 heartbeater.cc:503] Master 127.10.236.254:44197 was elected leader, sending a full tablet report...
I20260926 13:50:01.280086 12808 ts_tablet_manager.cc:1428] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 13:50:01.280908 12645 heartbeater.cc:503] Master 127.10.236.254:44197 was elected leader, sending a full tablet report...
I20260926 13:50:01.281638 12810 ts_tablet_manager.cc:1428] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:50:01.281032 12809 raft_consensus.cc:348] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.281764 12809 raft_consensus.cc:374] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.281832 12809 raft_consensus.cc:729] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea449e70c02649938a2618cf41add202, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.282523 12809 consensus_queue.cc:260] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.283141 12785 heartbeater.cc:503] Master 127.10.236.254:44197 was elected leader, sending a full tablet report...
I20260926 13:50:01.283533 12809 ts_tablet_manager.cc:1428] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260926 13:50:01.298118 12814 raft_consensus.cc:482] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:01.298316 12814 raft_consensus.cc:504] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.299708 12814 leader_election.cc:288] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077), ea449e70c02649938a2618cf41add202 (127.10.236.195:35179)
I20260926 13:50:01.305614 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.300108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:42949 (local address 127.10.236.195:35179)
0926 13:50:01.300243 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.300248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.300590 (+ 342us) server_negotiation.cc:407] Connection header received
0926 13:50:01.300739 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.300745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.300823 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.300912 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.301254 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.302066 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.304162 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.304806 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.304939 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.304991 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.305394 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.305477 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:50:01.305614 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.300181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44487 (local address 127.10.236.194:44077)
0926 13:50:01.300311 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.300315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.300614 (+ 299us) server_negotiation.cc:407] Connection header received
0926 13:50:01.300845 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.300851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.300910 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.300978 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.301323 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.302065 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.304162 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.304806 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.304939 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.304991 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.305394 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.305477 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:01.305680 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.299996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44077 (local address 127.10.236.193:44487)
0926 13:50:01.300502 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.300515 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.300676 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.301056 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.301059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.301076 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.301220 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.301227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.302214 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.302218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.303992 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.304005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.304950 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.304954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.305156 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.305238 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.305326 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.305470 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":436,"thread_start_us":99,"threads_started":1}
I20260926 13:50:01.305680 12817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.299995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35179 (local address 127.10.236.193:42949)
0926 13:50:01.300447 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.300463 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.300610 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.300934 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.300937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.300957 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.301085 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.301091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.302214 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.302218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.303991 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.304005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.304950 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.304954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.305151 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.305238 (+ 87us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.305337 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.305470 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":255,"thread_start_us":131,"threads_started":1}
I20260926 13:50:01.306468 12739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a8fdf843da46b3bf3b960b1f8ac512" candidate_uuid: "241a317b24b54e3998faa5afd6685643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea449e70c02649938a2618cf41add202" is_pre_election: true
I20260926 13:50:01.306468 12599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a8fdf843da46b3bf3b960b1f8ac512" candidate_uuid: "241a317b24b54e3998faa5afd6685643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc887bee4b4c1286dc7408c06edc34" is_pre_election: true
I20260926 13:50:01.306808 12739 raft_consensus.cc:2454] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 241a317b24b54e3998faa5afd6685643 in term 0.
I20260926 13:50:01.306808 12599 raft_consensus.cc:2454] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 241a317b24b54e3998faa5afd6685643 in term 0.
I20260926 13:50:01.307402 12393 leader_election.cc:302] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 241a317b24b54e3998faa5afd6685643, b7fc887bee4b4c1286dc7408c06edc34, ea449e70c02649938a2618cf41add202; no voters:
I20260926 13:50:01.307703 12814 raft_consensus.cc:2781] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:01.307789 12814 raft_consensus.cc:482] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:01.307832 12814 raft_consensus.cc:3037] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.308890 12814 raft_consensus.cc:504] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.309273 12814 leader_election.cc:288] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077), ea449e70c02649938a2618cf41add202 (127.10.236.195:35179)
I20260926 13:50:01.309633 12599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a8fdf843da46b3bf3b960b1f8ac512" candidate_uuid: "241a317b24b54e3998faa5afd6685643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc887bee4b4c1286dc7408c06edc34"
I20260926 13:50:01.309686 12739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a8fdf843da46b3bf3b960b1f8ac512" candidate_uuid: "241a317b24b54e3998faa5afd6685643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea449e70c02649938a2618cf41add202"
I20260926 13:50:01.309768 12599 raft_consensus.cc:3037] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.309890 12739 raft_consensus.cc:3037] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.311213 12739 raft_consensus.cc:2454] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 241a317b24b54e3998faa5afd6685643 in term 1.
I20260926 13:50:01.311233 12599 raft_consensus.cc:2454] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 241a317b24b54e3998faa5afd6685643 in term 1.
I20260926 13:50:01.311609 12391 leader_election.cc:302] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 241a317b24b54e3998faa5afd6685643, b7fc887bee4b4c1286dc7408c06edc34; no voters:
I20260926 13:50:01.311820 12814 raft_consensus.cc:2781] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:01.312062 12814 raft_consensus.cc:686] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 1 LEADER]: Becoming Leader. State: Replica: 241a317b24b54e3998faa5afd6685643, State: Running, Role: LEADER
I20260926 13:50:01.312445 12814 consensus_queue.cc:237] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.316020 12300 catalog_manager.cc:5495] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 reported cstate change: term changed from 0 to 1, leader changed from <none> to 241a317b24b54e3998faa5afd6685643 (127.10.236.193). New cstate: current_term: 1 leader_uuid: "241a317b24b54e3998faa5afd6685643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:01.345917 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.341578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.10.236.193:45161)
0926 13:50:01.341717 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.341720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.341734 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:01.341985 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.341988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.342045 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.342139 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:01.342475 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.343003 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.344026 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.344232 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.344308 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.344819 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:01.345303 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:01.345445 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.345606 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.345724 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:01.351935 12739 raft_consensus.cc:1261] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Refusing update from remote peer 241a317b24b54e3998faa5afd6685643: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:01.351938 12599 raft_consensus.cc:1261] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Refusing update from remote peer 241a317b24b54e3998faa5afd6685643: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:01.352638 12814 consensus_queue.cc:1035] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:01.352907 12819 consensus_queue.cc:1035] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:01.362478 12823 mvcc.cc:205] Tried to move back new op lower bound from 7333603743130062848 to 7333603742978502656. Current Snapshot: MvccSnapshot[applied={T|T < 7333603743130062848}]
I20260926 13:50:01.373594 12301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:01.374039 12301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:01.377059 12301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: Sending DeleteTablet for 3 replicas of tablet 09a8fdf843da46b3bf3b960b1f8ac512
I20260926 13:50:01.378049 12719 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a8fdf843da46b3bf3b960b1f8ac512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:01 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:40118
I20260926 13:50:01.378046 12439 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a8fdf843da46b3bf3b960b1f8ac512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:01 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:43350
I20260926 13:50:01.378275 12579 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a8fdf843da46b3bf3b960b1f8ac512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:01 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39354
I20260926 13:50:01.378635 12832 tablet_replica.cc:330] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: stopping tablet replica
I20260926 13:50:01.378834 12833 tablet_replica.cc:330] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: stopping tablet replica
I20260926 13:50:01.379033 12832 raft_consensus.cc:2229] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:01.379232 12833 raft_consensus.cc:2229] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:01.379544 12832 raft_consensus.cc:2258] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:01.379567 12301 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 13:50:01.379909 12833 raft_consensus.cc:2258] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 13:50:01.380002 12301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:01.380445 12831 tablet_replica.cc:330] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: stopping tablet replica
I20260926 13:50:01.380892 12831 raft_consensus.cc:2229] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:01.381235 12832 ts_tablet_manager.cc:1905] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:01.382042 12833 ts_tablet_manager.cc:1905] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:01.384562 12832 ts_tablet_manager.cc:1918] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:01.384687 12832 log.cc:1196] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/09a8fdf843da46b3bf3b960b1f8ac512
I20260926 13:50:01.385037 12832 ts_tablet_manager.cc:1939] T 09a8fdf843da46b3bf3b960b1f8ac512 P 241a317b24b54e3998faa5afd6685643: Deleting consensus metadata
I20260926 13:50:01.385185 12831 raft_consensus.cc:2258] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:01.385798 12439 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1b16b4107b4670928e7ee1cc48307e (DEFAULT_TABLE table=test-table [id=18468fd13709451e902d72bab5d06340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:01.386049 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1b16b4107b4670928e7ee1cc48307e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.386333 12287 catalog_manager.cc:4841] TS 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161): tablet 09a8fdf843da46b3bf3b960b1f8ac512 (table test-table [id=f9fba79496b34f87b1d0451e8855de5e]) successfully deleted
I20260926 13:50:01.387117 12831 ts_tablet_manager.cc:1905] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:01.388145 12833 ts_tablet_manager.cc:1918] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:01.388341 12808 tablet_bootstrap.cc:492] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Bootstrap starting.
I20260926 13:50:01.388334 12833 log.cc:1196] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/09a8fdf843da46b3bf3b960b1f8ac512
I20260926 13:50:01.388581 12719 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1b16b4107b4670928e7ee1cc48307e (DEFAULT_TABLE table=test-table [id=18468fd13709451e902d72bab5d06340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:01.388849 12719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1b16b4107b4670928e7ee1cc48307e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.388840 12833 ts_tablet_manager.cc:1939] T 09a8fdf843da46b3bf3b960b1f8ac512 P b7fc887bee4b4c1286dc7408c06edc34: Deleting consensus metadata
I20260926 13:50:01.389503 12808 tablet_bootstrap.cc:654] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.391044 12808 tablet_bootstrap.cc:492] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: No bootstrap required, opened a new log
I20260926 13:50:01.391150 12808 ts_tablet_manager.cc:1397] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:01.391247 12809 tablet_bootstrap.cc:492] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Bootstrap starting.
I20260926 13:50:01.391873 12831 ts_tablet_manager.cc:1918] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:01.391978 12831 log.cc:1196] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/09a8fdf843da46b3bf3b960b1f8ac512
I20260926 13:50:01.391836 12808 raft_consensus.cc:348] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.392136 12808 raft_consensus.cc:374] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.392230 12809 tablet_bootstrap.cc:654] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.392223 12808 raft_consensus.cc:729] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 241a317b24b54e3998faa5afd6685643, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.392356 12831 ts_tablet_manager.cc:1939] T 09a8fdf843da46b3bf3b960b1f8ac512 P ea449e70c02649938a2618cf41add202: Deleting consensus metadata
I20260926 13:50:01.392519 12808 consensus_queue.cc:260] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.393671 12289 catalog_manager.cc:4841] TS ea449e70c02649938a2618cf41add202 (127.10.236.195:35179): tablet 09a8fdf843da46b3bf3b960b1f8ac512 (table test-table [id=f9fba79496b34f87b1d0451e8855de5e]) successfully deleted
I20260926 13:50:01.393761 12287 catalog_manager.cc:4841] TS b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077): tablet 09a8fdf843da46b3bf3b960b1f8ac512 (table test-table [id=f9fba79496b34f87b1d0451e8855de5e]) successfully deleted
I20260926 13:50:01.394109 12808 ts_tablet_manager.cc:1428] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 13:50:01.394160 12579 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1b16b4107b4670928e7ee1cc48307e (DEFAULT_TABLE table=test-table [id=18468fd13709451e902d72bab5d06340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:01.394426 12579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1b16b4107b4670928e7ee1cc48307e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.396494 12809 tablet_bootstrap.cc:492] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: No bootstrap required, opened a new log
I20260926 13:50:01.396592 12809 ts_tablet_manager.cc:1397] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:50:01.396752 12810 tablet_bootstrap.cc:492] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Bootstrap starting.
I20260926 13:50:01.397118 12809 raft_consensus.cc:348] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.397226 12809 raft_consensus.cc:374] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.397253 12809 raft_consensus.cc:729] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea449e70c02649938a2618cf41add202, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.397436 12809 consensus_queue.cc:260] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.397694 12810 tablet_bootstrap.cc:654] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.397747 12809 ts_tablet_manager.cc:1428] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:01.398815 12810 tablet_bootstrap.cc:492] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: No bootstrap required, opened a new log
I20260926 13:50:01.398900 12810 ts_tablet_manager.cc:1397] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:01.399312 12810 raft_consensus.cc:348] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.399422 12810 raft_consensus.cc:374] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.399458 12810 raft_consensus.cc:729] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc887bee4b4c1286dc7408c06edc34, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.399596 12810 consensus_queue.cc:260] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.399858 12810 ts_tablet_manager.cc:1428] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260926 13:50:01.441659 12786 tablet.cc:2369] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:50:01.452433 12646 tablet.cc:2369] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:50:01.481470 12506 tablet.cc:2369] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:01.637229 12815 raft_consensus.cc:482] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:01.637389 12815 raft_consensus.cc:504] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.638661 12815 leader_election.cc:288] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161), ea449e70c02649938a2618cf41add202 (127.10.236.195:35179)
I20260926 13:50:01.642894 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.639066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:58781 (local address 127.10.236.195:35179)
0926 13:50:01.639296 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.639302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.639470 (+ 168us) server_negotiation.cc:407] Connection header received
0926 13:50:01.639696 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.639729 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.639827 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.639917 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.640202 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.640682 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.641879 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.642297 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.642404 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.642497 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.642681 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.642759 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260926 13:50:01.642953 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.638932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:45161 (local address 127.10.236.194:36383)
0926 13:50:01.639293 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.639309 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.639423 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.639768 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.639771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.639787 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.639923 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.639931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.640781 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.640784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.641750 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.641760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.642485 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.642489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.642586 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.642603 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.642699 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.642761 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":8,"thread_start_us":72,"threads_started":1}
I20260926 13:50:01.642946 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.639041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:36383 (local address 127.10.236.193:45161)
0926 13:50:01.639191 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.639199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.639402 (+ 203us) server_negotiation.cc:407] Connection header received
0926 13:50:01.639573 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.639578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.639644 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.639726 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.640035 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.640667 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.641922 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.642338 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.642490 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.642547 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.642734 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.642807 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:01.642973 12838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.638932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35179 (local address 127.10.236.194:58781)
0926 13:50:01.639343 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.639356 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.639559 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.639944 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.639947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.639962 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.640092 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.640101 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.640820 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.640824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.641745 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.641754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.642429 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.642435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.642539 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.642570 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.642652 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.642807 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":64,"thread_start_us":80,"threads_started":1}
I20260926 13:50:01.643445 12739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1b16b4107b4670928e7ee1cc48307e" candidate_uuid: "b7fc887bee4b4c1286dc7408c06edc34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea449e70c02649938a2618cf41add202" is_pre_election: true
I20260926 13:50:01.643512 12459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1b16b4107b4670928e7ee1cc48307e" candidate_uuid: "b7fc887bee4b4c1286dc7408c06edc34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "241a317b24b54e3998faa5afd6685643" is_pre_election: true
I20260926 13:50:01.643656 12739 raft_consensus.cc:2454] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc887bee4b4c1286dc7408c06edc34 in term 0.
I20260926 13:50:01.643792 12459 raft_consensus.cc:2454] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc887bee4b4c1286dc7408c06edc34 in term 0.
I20260926 13:50:01.644098 12533 leader_election.cc:302] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7fc887bee4b4c1286dc7408c06edc34, ea449e70c02649938a2618cf41add202; no voters:
I20260926 13:50:01.644400 12815 raft_consensus.cc:2781] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:01.644474 12815 raft_consensus.cc:482] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:01.644531 12815 raft_consensus.cc:3037] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.645805 12815 raft_consensus.cc:504] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.646231 12815 leader_election.cc:288] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [CANDIDATE]: Term 1 election: Requested vote from peers 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161), ea449e70c02649938a2618cf41add202 (127.10.236.195:35179)
I20260926 13:50:01.646621 12459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1b16b4107b4670928e7ee1cc48307e" candidate_uuid: "b7fc887bee4b4c1286dc7408c06edc34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "241a317b24b54e3998faa5afd6685643"
I20260926 13:50:01.646621 12739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1b16b4107b4670928e7ee1cc48307e" candidate_uuid: "b7fc887bee4b4c1286dc7408c06edc34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea449e70c02649938a2618cf41add202"
I20260926 13:50:01.646746 12459 raft_consensus.cc:3037] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.646746 12739 raft_consensus.cc:3037] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.647924 12739 raft_consensus.cc:2454] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc887bee4b4c1286dc7408c06edc34 in term 1.
I20260926 13:50:01.647924 12459 raft_consensus.cc:2454] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc887bee4b4c1286dc7408c06edc34 in term 1.
I20260926 13:50:01.648216 12531 leader_election.cc:302] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 241a317b24b54e3998faa5afd6685643, b7fc887bee4b4c1286dc7408c06edc34; no voters:
I20260926 13:50:01.648419 12815 raft_consensus.cc:2781] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:01.648607 12815 raft_consensus.cc:686] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc887bee4b4c1286dc7408c06edc34, State: Running, Role: LEADER
I20260926 13:50:01.648854 12815 consensus_queue.cc:237] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.651254 12301 catalog_manager.cc:5495] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "b7fc887bee4b4c1286dc7408c06edc34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } health_report { overall_health: UNKNOWN } } }
Sep 26 13:50:01 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430600, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:01.734760 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.725515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44197 (local address 127.0.0.1:60062)
0926 13:50:01.725885 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.725897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.726007 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.726296 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.726299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.726690 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:01.726788 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.726793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.727581 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.727584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.728208 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.728217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.728809 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.728812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.728896 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.729519 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:01.729534 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:01.731187 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:01.733071 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:01.733080 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:01.733086 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:01.733375 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:01.733689 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:01.733692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:01.733694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:01.733789 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:01.734188 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:01.734195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:01.734312 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.734418 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.734567 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":145,"threads_started":1}
I20260926 13:50:01.734880 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.725571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.10.236.254:44197)
0926 13:50:01.725708 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.725713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.725995 (+ 282us) server_negotiation.cc:407] Connection header received
0926 13:50:01.726108 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.726110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.726153 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.726249 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:01.726891 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.727422 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.728332 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.728614 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.728736 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.731345 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:01.731349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:01.731351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:01.731372 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:01.732957 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.733499 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.733502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.733503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.733577 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:01.733898 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:01.733900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:01.733902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:01.734089 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:01.734167 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.734468 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.734622 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":49}
I20260926 13:50:01.739012 12301 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:60062:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 13:50:01.739902 12301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:01.744937 12439 tablet_service.cc:1467] Processing CreateTablet for tablet f46e44cb5f104a19958137d5f16632a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49f10feede3d40809496c170e523f9d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:01.744974 12579 tablet_service.cc:1467] Processing CreateTablet for tablet f46e44cb5f104a19958137d5f16632a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49f10feede3d40809496c170e523f9d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:01.745335 12579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46e44cb5f104a19958137d5f16632a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.745335 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46e44cb5f104a19958137d5f16632a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.747421 12719 tablet_service.cc:1467] Processing CreateTablet for tablet f46e44cb5f104a19958137d5f16632a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49f10feede3d40809496c170e523f9d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:01.747872 12719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46e44cb5f104a19958137d5f16632a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:01.748159 12808 tablet_bootstrap.cc:492] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: Bootstrap starting.
I20260926 13:50:01.748190 12810 tablet_bootstrap.cc:492] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: Bootstrap starting.
I20260926 13:50:01.748980 12808 tablet_bootstrap.cc:654] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.749109 12810 tablet_bootstrap.cc:654] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.750119 12808 tablet_bootstrap.cc:492] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: No bootstrap required, opened a new log
I20260926 13:50:01.750211 12808 ts_tablet_manager.cc:1397] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:01.750598 12808 raft_consensus.cc:348] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.750731 12808 raft_consensus.cc:374] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.750772 12808 raft_consensus.cc:729] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 241a317b24b54e3998faa5afd6685643, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.750891 12808 consensus_queue.cc:260] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.750978 12809 tablet_bootstrap.cc:492] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: Bootstrap starting.
I20260926 13:50:01.751209 12808 ts_tablet_manager.cc:1428] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:01.751398 12814 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:01.751539 12814 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:01.751636 12810 tablet_bootstrap.cc:492] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: No bootstrap required, opened a new log
I20260926 13:50:01.751708 12810 ts_tablet_manager.cc:1397] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 13:50:01.752003 12809 tablet_bootstrap.cc:654] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:01.752182 12810 raft_consensus.cc:348] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.752300 12810 raft_consensus.cc:374] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.752341 12810 raft_consensus.cc:729] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc887bee4b4c1286dc7408c06edc34, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.752458 12810 consensus_queue.cc:260] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.752832 12810 ts_tablet_manager.cc:1428] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:01.753108 12815 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:01.753232 12815 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:01.753551 12809 tablet_bootstrap.cc:492] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: No bootstrap required, opened a new log
I20260926 13:50:01.753633 12809 ts_tablet_manager.cc:1397] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:50:01.754037 12809 raft_consensus.cc:348] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.754161 12809 raft_consensus.cc:374] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:01.754199 12809 raft_consensus.cc:729] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea449e70c02649938a2618cf41add202, State: Initialized, Role: FOLLOWER
I20260926 13:50:01.754348 12809 consensus_queue.cc:260] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.754614 12809 ts_tablet_manager.cc:1428] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:50:01.754602 12816 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:01.754765 12816 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:01.975101 12816 raft_consensus.cc:482] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:01.975247 12816 raft_consensus.cc:504] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.976368 12816 leader_election.cc:288] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161), b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077)
I20260926 13:50:01.980865 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.976866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:41049 (local address 127.10.236.193:45161)
0926 13:50:01.977035 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.977039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.977101 (+ 62us) server_negotiation.cc:407] Connection header received
0926 13:50:01.977361 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.977366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.977424 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.977502 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.977849 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.978421 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.979828 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.980208 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.980309 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.980389 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.980634 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.980720 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:50:01.980901 12852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.977207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:38403 (local address 127.10.236.194:44077)
0926 13:50:01.977453 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:01.977456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:01.977493 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:50:01.977549 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:01.977552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:01.977597 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:01.977691 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:01.977972 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.978592 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.979760 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.980238 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.980387 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.980441 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:01.980675 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 13:50:01.980751 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
I20260926 13:50:01.980927 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.976708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:45161 (local address 127.10.236.195:41049)
0926 13:50:01.976976 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.976991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.977154 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.977548 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.977552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.977572 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.977720 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.977728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.978531 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.978534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.979671 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.979680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.980338 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.980344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.980441 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.980531 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.980596 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.980739 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":91,"thread_start_us":63,"threads_started":1}
I20260926 13:50:01.981145 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:01.977067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44077 (local address 127.10.236.195:38403)
0926 13:50:01.977322 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:01.977335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:01.977418 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:01.977705 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:01.977707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:01.977722 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:01.977853 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.977860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.978705 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.978707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.979635 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:01.979644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:01.980384 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:01.980388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:01.980526 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:01.980544 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:50:01.980633 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 13:50:01.980723 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":53,"thread_start_us":85,"threads_started":1}
I20260926 13:50:01.981485 12459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46e44cb5f104a19958137d5f16632a1" candidate_uuid: "ea449e70c02649938a2618cf41add202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "241a317b24b54e3998faa5afd6685643" is_pre_election: true
I20260926 13:50:01.981638 12599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46e44cb5f104a19958137d5f16632a1" candidate_uuid: "ea449e70c02649938a2618cf41add202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc887bee4b4c1286dc7408c06edc34" is_pre_election: true
I20260926 13:50:01.981680 12459 raft_consensus.cc:2454] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea449e70c02649938a2618cf41add202 in term 0.
I20260926 13:50:01.981792 12599 raft_consensus.cc:2454] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea449e70c02649938a2618cf41add202 in term 0.
I20260926 13:50:01.982136 12671 leader_election.cc:302] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 241a317b24b54e3998faa5afd6685643, ea449e70c02649938a2618cf41add202; no voters:
I20260926 13:50:01.982429 12816 raft_consensus.cc:2781] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:01.982493 12816 raft_consensus.cc:482] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:01.982542 12816 raft_consensus.cc:3037] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.983567 12816 raft_consensus.cc:504] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.983934 12816 leader_election.cc:288] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [CANDIDATE]: Term 1 election: Requested vote from peers 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161), b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077)
I20260926 13:50:01.984335 12599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46e44cb5f104a19958137d5f16632a1" candidate_uuid: "ea449e70c02649938a2618cf41add202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc887bee4b4c1286dc7408c06edc34"
I20260926 13:50:01.984360 12459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46e44cb5f104a19958137d5f16632a1" candidate_uuid: "ea449e70c02649938a2618cf41add202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "241a317b24b54e3998faa5afd6685643"
I20260926 13:50:01.984478 12459 raft_consensus.cc:3037] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.984483 12599 raft_consensus.cc:3037] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:01.985399 12459 raft_consensus.cc:2454] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea449e70c02649938a2618cf41add202 in term 1.
I20260926 13:50:01.985585 12599 raft_consensus.cc:2454] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea449e70c02649938a2618cf41add202 in term 1.
I20260926 13:50:01.985671 12671 leader_election.cc:302] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 241a317b24b54e3998faa5afd6685643, ea449e70c02649938a2618cf41add202; no voters:
I20260926 13:50:01.985949 12816 raft_consensus.cc:2781] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:01.986145 12816 raft_consensus.cc:686] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [term 1 LEADER]: Becoming Leader. State: Replica: ea449e70c02649938a2618cf41add202, State: Running, Role: LEADER
I20260926 13:50:01.986414 12816 consensus_queue.cc:237] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } }
I20260926 13:50:01.987648 12853 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: TxnStatusTablet state changed. Reason: New leader ea449e70c02649938a2618cf41add202. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:01.987761 12853 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:01.988045 12855 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 13:50:01.988199 12855 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 13:50:01.988952 12302 catalog_manager.cc:5495] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea449e70c02649938a2618cf41add202 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } health_report { overall_health: HEALTHY } } }
I20260926 13:50:02.051720 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.047170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35179 (local address 127.0.0.1:40126)
0926 13:50:02.047347 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:02.047364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:02.047465 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:02.047909 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:02.047912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:02.047920 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:02.048031 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:02.048036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.049084 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.049087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:02.049713 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:02.049722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.050325 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.050328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:02.050397 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.050441 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.051349 (+ 908us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.051366 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 13:50:02.051468 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 13:50:02.051612 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 13:50:02.051716 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.047311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.10.236.195:35179)
0926 13:50:02.047428 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:02.047433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:02.047450 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:50:02.047657 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:02.047660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:02.047732 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:02.047844 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:02.048148 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.048956 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.049829 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.050194 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.050310 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.050581 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.051116 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.051236 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:02.051509 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 13:50:02.051569 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:50:02.054869 12599 raft_consensus.cc:1261] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Refusing update from remote peer ea449e70c02649938a2618cf41add202: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:02.055044 12459 raft_consensus.cc:1261] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Refusing update from remote peer ea449e70c02649938a2618cf41add202: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:02.055583 12816 consensus_queue.cc:1035] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:02.055773 12853 consensus_queue.cc:1035] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:02.057263 12815 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: TxnStatusTablet state changed. Reason: New leader ea449e70c02649938a2618cf41add202. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:02.057392 12815 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:02.057438 12814 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: TxnStatusTablet state changed. Reason: New leader ea449e70c02649938a2618cf41add202. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:02.057542 12814 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:02.060330 12853 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:02.060477 12853 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:02.061136 12814 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:02.061255 12814 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P 241a317b24b54e3998faa5afd6685643: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:02.061777 12816 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:02.061884 12816 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P ea449e70c02649938a2618cf41add202: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:02.061960 12815 tablet_replica.cc:439] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea449e70c02649938a2618cf41add202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 } } peers { permanent_uuid: "b7fc887bee4b4c1286dc7408c06edc34" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44077 } } peers { permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 } } }
I20260926 13:50:02.062052 12815 tablet_replica.cc:442] T f46e44cb5f104a19958137d5f16632a1 P b7fc887bee4b4c1286dc7408c06edc34: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:02.062784 12859 mvcc.cc:205] Tried to move back new op lower bound from 7333603746011521024 to 7333603745738629120. Current Snapshot: MvccSnapshot[applied={T|T < 7333603746011521024}]
I20260926 13:50:02.072819 12852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.068332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.10.236.194:44077)
0926 13:50:02.068468 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:02.068472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:02.068738 (+ 266us) server_negotiation.cc:407] Connection header received
0926 13:50:02.068869 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:02.068873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:02.068934 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:02.069041 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:02.069247 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.070016 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.071111 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.071464 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.071592 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.071832 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.072386 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.072496 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:02.072626 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 13:50:02.072682 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:50:02.083801 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.080014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:47495 (local address 127.10.236.195:35179)
0926 13:50:02.080158 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:02.080165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:02.080407 (+ 242us) server_negotiation.cc:407] Connection header received
0926 13:50:02.080539 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:02.080542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:02.080594 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:02.080677 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:02.081288 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.081790 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.082671 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.082901 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.083021 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.083308 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.083382 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.083478 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:02.083600 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 13:50:02.083650 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 13:50:02.083817 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.079914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35179 (local address 127.10.236.194:47495)
0926 13:50:02.080315 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:02.080326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:02.080446 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:02.080688 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:02.080691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:02.080699 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:02.081152 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:02.081159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.081906 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.081910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:02.082548 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:02.082556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.083032 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.083035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:02.083131 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.083169 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.083521 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.083528 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:50:02.083590 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 13:50:02.083668 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":129,"threads_started":1}
I20260926 13:50:02.090034 12739 raft_consensus.cc:1261] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Refusing update from remote peer b7fc887bee4b4c1286dc7408c06edc34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:02.090508 12815 consensus_queue.cc:1035] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea449e70c02649938a2618cf41add202" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:02.091246 12459 raft_consensus.cc:1261] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Refusing update from remote peer b7fc887bee4b4c1286dc7408c06edc34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:02.091612 12815 consensus_queue.cc:1035] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "241a317b24b54e3998faa5afd6685643" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 45161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:02.096560 12860 mvcc.cc:205] Tried to move back new op lower bound from 7333603746156494848 to 7333603744356024320. Current Snapshot: MvccSnapshot[applied={T|T < 7333603746156494848}]
I20260926 13:50:02.115947 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.111200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44077 (local address 127.10.236.195:52769)
0926 13:50:02.111871 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:02.111885 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:02.112021 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:02.112306 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:02.112309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:02.112317 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:02.112480 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:02.112488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.113491 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.113495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:02.114526 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:02.114536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.115058 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.115062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:02.115165 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.115209 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.115587 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.115597 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 13:50:02.115666 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:50:02.115747 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":148,"threads_started":1}
I20260926 13:50:02.115904 12852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.111630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:52769 (local address 127.10.236.194:44077)
0926 13:50:02.111791 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:02.111796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:02.111974 (+ 178us) server_negotiation.cc:407] Connection header received
0926 13:50:02.112137 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:02.112141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:02.112201 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:02.112294 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:02.112618 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.113217 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.114664 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.114922 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.115047 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.115378 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:02.115464 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:02.115584 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:02.115685 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 13:50:02.115741 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:50:02.146028 12300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:02.146221 12300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:02.148645 12300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8e7b66c93ba48afbaaf1c90181706a7: Sending DeleteTablet for 3 replicas of tablet 6f1b16b4107b4670928e7ee1cc48307e
I20260926 13:50:02.149358 12719 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1b16b4107b4670928e7ee1cc48307e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:02 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:40118
I20260926 13:50:02.149386 12439 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1b16b4107b4670928e7ee1cc48307e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:02 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:43350
I20260926 13:50:02.149993 12579 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1b16b4107b4670928e7ee1cc48307e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:02 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39354
I20260926 13:50:02.150388 12881 tablet_replica.cc:330] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: stopping tablet replica
I20260926 13:50:02.150574 12881 raft_consensus.cc:2229] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:02.150974 12881 raft_consensus.cc:2258] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:02.151845 12881 ts_tablet_manager.cc:1905] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:02.152097 12880 tablet_replica.cc:330] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: stopping tablet replica
I20260926 13:50:02.152284 12880 raft_consensus.cc:2229] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:02.152463 12880 raft_consensus.cc:2258] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:02.153992 12880 ts_tablet_manager.cc:1905] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:02.156687 12879 tablet_replica.cc:330] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: stopping tablet replica
I20260926 13:50:02.156890 12879 raft_consensus.cc:2229] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:02.157069 12879 raft_consensus.cc:2258] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:02.160054 12879 ts_tablet_manager.cc:1905] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 13:50:02 dist-test-slave-r34j krb5kdc[12257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:50:02.161667 12881 ts_tablet_manager.cc:1918] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:02.164777 12879 ts_tablet_manager.cc:1918] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:02.164963 12881 log.cc:1196] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/6f1b16b4107b4670928e7ee1cc48307e
I20260926 13:50:02.165123 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.151204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.10.236.193:45161)
0926 13:50:02.151321 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:02.151326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:02.151888 (+ 562us) server_negotiation.cc:407] Connection header received
0926 13:50:02.152496 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:02.152500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:02.152555 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:02.152611 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:02.153868 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.154676 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.156070 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.156398 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.156499 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.160895 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:02.160908 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:02.160923 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:02.160954 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:02.162977 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:02.163545 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:02.163555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:02.163559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:02.163664 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:02.163975 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:02.163979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:02.163981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:02.164277 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:02.164464 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:02.164813 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 13:50:02.164940 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:50:02.165359 12881 ts_tablet_manager.cc:1939] T 6f1b16b4107b4670928e7ee1cc48307e P b7fc887bee4b4c1286dc7408c06edc34: Deleting consensus metadata
I20260926 13:50:02.166225 12287 catalog_manager.cc:4841] TS b7fc887bee4b4c1286dc7408c06edc34 (127.10.236.194:44077): tablet 6f1b16b4107b4670928e7ee1cc48307e (table test-table [id=18468fd13709451e902d72bab5d06340]) successfully deleted
W20260926 13:50:02.166358 12499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43372
I20260926 13:50:02.166623 12880 ts_tablet_manager.cc:1918] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:02.166796 12880 log.cc:1196] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/6f1b16b4107b4670928e7ee1cc48307e
I20260926 13:50:02.167142 12880 ts_tablet_manager.cc:1939] T 6f1b16b4107b4670928e7ee1cc48307e P 241a317b24b54e3998faa5afd6685643: Deleting consensus metadata
I20260926 13:50:02.168066 12287 catalog_manager.cc:4841] TS 241a317b24b54e3998faa5afd6685643 (127.10.236.193:45161): tablet 6f1b16b4107b4670928e7ee1cc48307e (table test-table [id=18468fd13709451e902d72bab5d06340]) successfully deleted
I20260926 13:50:02.168340 12879 log.cc:1196] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/6f1b16b4107b4670928e7ee1cc48307e
I20260926 13:50:02.168702 12879 ts_tablet_manager.cc:1939] T 6f1b16b4107b4670928e7ee1cc48307e P ea449e70c02649938a2618cf41add202: Deleting consensus metadata
I20260926 13:50:02.169515 12289 catalog_manager.cc:4841] TS ea449e70c02649938a2618cf41add202 (127.10.236.195:35179): tablet 6f1b16b4107b4670928e7ee1cc48307e (table test-table [id=18468fd13709451e902d72bab5d06340]) successfully deleted
I20260926 13:50:02.178196 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:02.169098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.10.236.254:44197)
0926 13:50:02.169219 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:02.169223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:02.169697 (+ 474us) server_negotiation.cc:407] Connection header received
0926 13:50:02.169912 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:02.169916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:02.169972 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:02.170068 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:02.170928 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.171758 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.172796 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:02.173139 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:02.173256 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:02.174635 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:02.174639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:02.174649 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:02.174677 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:02.176472 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:02.176877 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:02.176880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:02.176883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:02.176975 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:02.177225 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:02.177228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:02.177230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:02.177453 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:02.177593 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:02.177886 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 13:50:02.178005 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 13:50:02.178682 12300 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60078
I20260926 13:50:02.180207 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 12374
I20260926 13:50:02.189459 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 12514
I20260926 13:50:02.195653 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 12654
I20260926 13:50:02.202833 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 12270
2026-09-26T13:50:02Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3157 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:02 dist-test-slave-r34j krb5kdc[12888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:02 dist-test-slave-r34j krb5kdc[12888](info): set up 2 sockets
Sep 26 13:50:02 dist-test-slave-r34j krb5kdc[12888](info): commencing operation
krb5kdc: starting...
W20260926 13:50:03.271313 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:03 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:03.404186 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37681
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:37681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:03.517764 12901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:03.518059 12901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:03.518100 12901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:03.522205 12901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:03.522275 12901 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 13:50:03.522293 12901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:03.522311 12901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:03.522327 12901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:03.522343 12901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:03.527915 12901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:37681
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:03.529035 12901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:03.530104 12901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:03.538868 12909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:03.538868 12906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:03.538928 12907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:03.539076 12901 server_base.cc:1034] running on GCE node
I20260926 13:50:03.539892 12901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:03.541019 12901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:03.542191 12901 hybrid_clock.cc:648] HybridClock initialized: now 1790430603542172 us; error 33 us; skew 500 ppm
Sep 26 13:50:03 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:03.545255 12901 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:03.546190 12901 webserver.cc:460] Webserver started at http://127.10.236.254:45989/ using document root <none> and password file <none>
I20260926 13:50:03.546768 12901 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:03.546833 12901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:03.547048 12901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:03.549199 12901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "5102595320224e10a7a553c98aa450a6"
format_stamp: "Formatted at 2026-09-26 13:50:03 on dist-test-slave-r34j"
server_key: "0d553c63c13096359f24ab9c2b213107"
server_key_iv: "7b15dd6476e6d1e9667c78675cacccf4"
server_key_version: "encryptionkey@0"
I20260926 13:50:03.549780 12901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "5102595320224e10a7a553c98aa450a6"
format_stamp: "Formatted at 2026-09-26 13:50:03 on dist-test-slave-r34j"
server_key: "0d553c63c13096359f24ab9c2b213107"
server_key_iv: "7b15dd6476e6d1e9667c78675cacccf4"
server_key_version: "encryptionkey@0"
I20260926 13:50:03.553674 12901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260926 13:50:03.556344 12916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:03.557758 12901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:50:03.557911 12901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "5102595320224e10a7a553c98aa450a6"
format_stamp: "Formatted at 2026-09-26 13:50:03 on dist-test-slave-r34j"
server_key: "0d553c63c13096359f24ab9c2b213107"
server_key_iv: "7b15dd6476e6d1e9667c78675cacccf4"
server_key_version: "encryptionkey@0"
I20260926 13:50:03.558036 12901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:03.606524 12901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:03.610121 12901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:03.610354 12901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:03.618487 12901 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:37681
I20260926 13:50:03.618501 12978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:37681 every 8 connection(s)
I20260926 13:50:03.619459 12901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:03.622148 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 12901
I20260926 13:50:03.622277 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:03.622527 12979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:03.622555 11187 external_mini_cluster.cc:1428] Setting key 277f1649eb1abc1fb50e81b6010b1b2d
I20260926 13:50:03.629287 12979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Bootstrap starting.
I20260926 13:50:03.631771 12979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Neither blocks nor log segments found. Creating new log.
Sep 26 13:50:03 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:03.632548 12979 log.cc:824] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:03.634593 12979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: No bootstrap required, opened a new log
I20260926 13:50:03.637722 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:03.624049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.10.236.254:37681)
0926 13:50:03.624502 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:03.624517 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:03.624553 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:50:03.625415 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:03.625434 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:03.625808 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:03.626177 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:03.626961 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.628033 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.629117 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.629687 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.629860 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:03.632558 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:03.632563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:03.632577 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:03.632603 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:03.634864 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:03.635319 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:03.635331 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:03.635336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:03.635453 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:03.635796 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:03.635800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:03.635801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:03.636182 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:03.636383 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:03.636665 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 13:50:03.636998 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260926 13:50:03.637648 12979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5102595320224e10a7a553c98aa450a6" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37681 } }
I20260926 13:50:03.637948 12979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:03.638029 12979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5102595320224e10a7a553c98aa450a6, State: Initialized, Role: FOLLOWER
I20260926 13:50:03.638448 12979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5102595320224e10a7a553c98aa450a6" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37681 } }
I20260926 13:50:03.638597 12979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:03.638679 12979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:03.639601 12979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:03.640673 12979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5102595320224e10a7a553c98aa450a6" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37681 } }
I20260926 13:50:03.641023 12979 leader_election.cc:302] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5102595320224e10a7a553c98aa450a6; no voters:
I20260926 13:50:03.641355 12979 leader_election.cc:288] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:03.641481 12984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:03.641692 12984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [term 1 LEADER]: Becoming Leader. State: Replica: 5102595320224e10a7a553c98aa450a6, State: Running, Role: LEADER
I20260926 13:50:03.642071 12984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5102595320224e10a7a553c98aa450a6" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37681 } }
I20260926 13:50:03.642587 12979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:03.644014 12985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5102595320224e10a7a553c98aa450a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5102595320224e10a7a553c98aa450a6" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37681 } } }
I20260926 13:50:03.644032 12986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5102595320224e10a7a553c98aa450a6. Latest consensus state: current_term: 1 leader_uuid: "5102595320224e10a7a553c98aa450a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5102595320224e10a7a553c98aa450a6" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37681 } } }
I20260926 13:50:03.644141 12986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:03.644141 12985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:03.644579 12995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:03.647210 12995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:03.652184 12995 catalog_manager.cc:1347] Generated new cluster ID: baca71e827b14143b1386bf04776aaf7
I20260926 13:50:03.652251 12995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:03.661263 12995 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:03.662412 12995 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:03.667838 12995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Generated new TSK 0
I20260926 13:50:03.668494 12995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:03.699365 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37681
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:03.814848 13005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:03.815207 13005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:03.815296 13005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:03.819763 13005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:03.819913 13005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:03.819996 13005 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 13:50:03.820044 13005 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:03.820122 13005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:03.825851 13005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:37681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:03.826999 13005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:03.828085 13005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:03.837777 13005 server_base.cc:1034] running on GCE node
W20260926 13:50:03.837697 13011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:03.837693 13013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:03.837700 13010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:03.838402 13005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:03.839046 13005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:03.840226 13005 hybrid_clock.cc:648] HybridClock initialized: now 1790430603840209 us; error 29 us; skew 500 ppm
Sep 26 13:50:03 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:03.842989 13005 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:03.844096 13005 webserver.cc:460] Webserver started at http://127.10.236.193:38747/ using document root <none> and password file <none>
I20260926 13:50:03.844691 13005 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:03.844771 13005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:03.845016 13005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:03.847052 13005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "39c58b4536044fcf8e5597ca32a26a10"
format_stamp: "Formatted at 2026-09-26 13:50:03 on dist-test-slave-r34j"
server_key: "ae2a6c89f6c5285a0810d3a1af8b8da6"
server_key_iv: "025a8f25fc080963fe6dffd6db3e3025"
server_key_version: "encryptionkey@0"
I20260926 13:50:03.847645 13005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "39c58b4536044fcf8e5597ca32a26a10"
format_stamp: "Formatted at 2026-09-26 13:50:03 on dist-test-slave-r34j"
server_key: "ae2a6c89f6c5285a0810d3a1af8b8da6"
server_key_iv: "025a8f25fc080963fe6dffd6db3e3025"
server_key_version: "encryptionkey@0"
I20260926 13:50:03.851250 13005 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:03.853824 13020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:03.855182 13005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 13:50:03.855350 13005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "39c58b4536044fcf8e5597ca32a26a10"
format_stamp: "Formatted at 2026-09-26 13:50:03 on dist-test-slave-r34j"
server_key: "ae2a6c89f6c5285a0810d3a1af8b8da6"
server_key_iv: "025a8f25fc080963fe6dffd6db3e3025"
server_key_version: "encryptionkey@0"
I20260926 13:50:03.855475 13005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:03.870280 13005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:03.873780 13005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:03.874004 13005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:03.875903 13005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:03.875980 13005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:03.876052 13005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:03.876099 13005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:50:03 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:03.887250 13005 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:36421
I20260926 13:50:03.887441 13135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:36421 every 8 connection(s)
I20260926 13:50:03.888242 13005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:03.890098 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:03.876792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:55527 (local address 127.10.236.254:37681)
0926 13:50:03.876917 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:03.876921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:03.878073 (+ 1152us) server_negotiation.cc:407] Connection header received
0926 13:50:03.879275 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:03.879278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:03.879340 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:03.879447 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:03.880915 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.881455 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.882631 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.882906 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.883031 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:03.886125 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:03.886129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:03.886134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:03.886171 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:03.887928 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:03.888473 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:03.888477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:03.888479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:03.888565 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:03.888917 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:03.888920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:03.888922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:03.889149 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:03.889245 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:03.889836 (+ 591us) server_negotiation.cc:299] Negotiation successful
0926 13:50:03.889979 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:50:03.890933 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:03.877282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.10.236.193:55527)
0926 13:50:03.877840 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:03.877883 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:03.879035 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:03.879649 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:03.879656 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:03.880093 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:03.880753 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:03.880768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.881586 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.881590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:03.882494 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:03.882502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.883082 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.883086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:03.883312 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:03.884091 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:03.884107 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:03.885945 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:03.888051 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:03.888056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:03.888072 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:03.888382 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:03.888660 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:03.888664 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:03.888667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:03.888812 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:03.889297 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:03.889303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:03.889575 (+ 272us) client_negotiation.cc:769] Sending connection context
0926 13:50:03.889781 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 13:50:03.890114 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":138,"threads_started":1}
I20260926 13:50:03.892210 13136 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37681
I20260926 13:50:03.892371 13136 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:03.892958 13136 heartbeater.cc:511] Master 127.10.236.254:37681 requested a full tablet report, sending...
I20260926 13:50:03.894552 12932 ts_manager.cc:194] Registered new tserver with Master: 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421)
I20260926 13:50:03.896068 12932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:55527
I20260926 13:50:03.896291 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13005
I20260926 13:50:03.896446 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:03.896780 11187 external_mini_cluster.cc:1428] Setting key 840046a3dcef0270223af98b85a1a78c
I20260926 13:50:03.903076 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:03.894404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.10.236.193:54533)
0926 13:50:03.894670 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:03.894706 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:03.894810 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:03.895103 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:03.895106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:03.895306 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:03.895418 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:03.895424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.896485 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.896488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:03.897325 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:03.897335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.897900 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.897904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:03.898085 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:03.898693 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:03.898709 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:03.899240 (+ 531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:03.901403 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:03.901415 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:03.901417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:03.901613 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:03.901960 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:03.901962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:03.901963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:03.902044 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:03.902567 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:03.902570 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:03.902654 (+ 84us) client_negotiation.cc:769] Sending connection context
0926 13:50:03.902745 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 13:50:03.902870 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260926 13:50:03.903199 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:03.894502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:54533 (local address 127.10.236.254:37681)
0926 13:50:03.894630 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:03.894634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:03.894784 (+ 150us) server_negotiation.cc:407] Connection header received
0926 13:50:03.894892 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:03.894895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:03.894936 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:03.895013 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:03.895608 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.896337 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.897442 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:03.897769 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:03.897899 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:03.899401 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:03.899404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:03.899406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:03.899428 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:03.901278 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:03.901718 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:03.901725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:03.901730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:03.901828 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:03.902139 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:03.902146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:03.902151 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:03.902442 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:03.902557 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:03.902835 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 13:50:03.902984 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:03.928143 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37681
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:04.045993 13145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:04.046270 13145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:04.046319 13145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:04.050417 13145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:04.050484 13145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:04.050524 13145 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 13:50:04.050539 13145 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:04.050643 13145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:04.056268 13145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:37681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:04.057363 13145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:04.058456 13145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:04.068641 13150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:04.068640 13151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:04.068876 13145 server_base.cc:1034] running on GCE node
W20260926 13:50:04.068657 13153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:04.069341 13145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:04.070096 13145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:04.071273 13145 hybrid_clock.cc:648] HybridClock initialized: now 1790430604071254 us; error 32 us; skew 500 ppm
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430604, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:04.074147 13145 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:04.075109 13145 webserver.cc:460] Webserver started at http://127.10.236.194:39473/ using document root <none> and password file <none>
I20260926 13:50:04.075804 13145 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:04.075874 13145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:04.076054 13145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:04.078007 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "543647d1a5eb43f39199da4bb243585c"
format_stamp: "Formatted at 2026-09-26 13:50:04 on dist-test-slave-r34j"
server_key: "69883f166732588ce07b1bd18986e171"
server_key_iv: "45b9b4e2bf334dff863ae7924e2eada2"
server_key_version: "encryptionkey@0"
I20260926 13:50:04.078495 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "543647d1a5eb43f39199da4bb243585c"
format_stamp: "Formatted at 2026-09-26 13:50:04 on dist-test-slave-r34j"
server_key: "69883f166732588ce07b1bd18986e171"
server_key_iv: "45b9b4e2bf334dff863ae7924e2eada2"
server_key_version: "encryptionkey@0"
I20260926 13:50:04.082182 13145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 13:50:04.085040 13160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:04.086838 13145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:50:04.086941 13145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "543647d1a5eb43f39199da4bb243585c"
format_stamp: "Formatted at 2026-09-26 13:50:04 on dist-test-slave-r34j"
server_key: "69883f166732588ce07b1bd18986e171"
server_key_iv: "45b9b4e2bf334dff863ae7924e2eada2"
server_key_version: "encryptionkey@0"
I20260926 13:50:04.087036 13145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:04.115252 13145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:04.119048 13145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:04.119288 13145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:04.120880 13145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:04.120936 13145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:04.121009 13145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:04.121047 13145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430604, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:04.132638 13145 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:35027
I20260926 13:50:04.132666 13275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:35027 every 8 connection(s)
I20260926 13:50:04.133692 13145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:04.135564 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13145
I20260926 13:50:04.135715 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:04.136085 11187 external_mini_cluster.cc:1428] Setting key 43a2153c4d1872a6ca5131fba3accb5b
I20260926 13:50:04.137287 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.121954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:59183 (local address 127.10.236.254:37681)
0926 13:50:04.122087 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.122091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.123259 (+ 1168us) server_negotiation.cc:407] Connection header received
0926 13:50:04.124439 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.124442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.124502 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.124614 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:04.126441 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.127311 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.129506 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.129725 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.129870 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.133263 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:04.133273 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:04.133275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:04.133301 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:04.134914 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.135525 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.135530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.135532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.135618 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.136024 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.136027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.136029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.136276 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:04.136384 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.136987 (+ 603us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.137108 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:50:04.138159 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.122528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.10.236.194:59183)
0926 13:50:04.123080 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.123123 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.124122 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.124874 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.124887 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.125450 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:04.126239 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.126268 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.127467 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.127475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.129333 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.129345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.129951 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.129955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.130214 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.131079 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:04.131108 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:04.133076 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:04.135028 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.135034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.135049 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.135392 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.135766 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.135772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.135775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.135919 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.136381 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:04.136388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:04.136704 (+ 316us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.136925 (+ 221us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.137241 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":160,"threads_started":1}
I20260926 13:50:04.139446 13276 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37681
I20260926 13:50:04.139631 13276 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:04.140277 13276 heartbeater.cc:511] Master 127.10.236.254:37681 requested a full tablet report, sending...
I20260926 13:50:04.141538 12933 ts_manager.cc:194] Registered new tserver with Master: 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027)
I20260926 13:50:04.142364 12933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:59183
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:04.151029 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.141769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:60883 (local address 127.10.236.254:37681)
0926 13:50:04.142699 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.142704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.142716 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:50:04.142748 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.142750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.142791 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.142873 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:04.143581 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.144228 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.145743 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.145963 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.146050 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.147598 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:04.147601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:04.147603 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:04.147625 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:04.149133 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.149652 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.149654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.149656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.149735 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.150094 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.150097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.150098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.150349 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:04.150436 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.150797 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.150911 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836}
I20260926 13:50:04.151149 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.141803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.10.236.194:60883)
0926 13:50:04.142112 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.142126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.142207 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.142902 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.142908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.143183 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:04.143337 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.143346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.144357 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.144361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.145443 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.145458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.146091 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.146098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.146242 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.146840 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:04.146858 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:04.147381 (+ 523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:04.149253 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.149260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.149262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.149533 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.149852 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.149855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.149857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.149965 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.150464 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:04.150468 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:04.150578 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.150722 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.150900 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:04.172063 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37681
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:04.283397 13285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:04.283747 13285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:04.283795 13285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:04.287834 13285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:04.287890 13285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:04.287930 13285 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 13:50:04.287945 13285 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:04.288012 13285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:04.293473 13285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:37681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:04.294543 13285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:04.295657 13285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:04.305408 13290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:04.305436 13293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:04.305513 13285 server_base.cc:1034] running on GCE node
W20260926 13:50:04.305411 13291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:04.306105 13285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:04.306748 13285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:04.307911 13285 hybrid_clock.cc:648] HybridClock initialized: now 1790430604307900 us; error 32 us; skew 500 ppm
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430604, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:04.310708 13285 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:04.311656 13285 webserver.cc:460] Webserver started at http://127.10.236.195:33165/ using document root <none> and password file <none>
I20260926 13:50:04.312217 13285 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:04.312275 13285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:04.312489 13285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:04.314456 13285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "3625c8f58abd4e1694562858b4a841d8"
format_stamp: "Formatted at 2026-09-26 13:50:04 on dist-test-slave-r34j"
server_key: "44b2e0723c4d7576831de0ed83071b4b"
server_key_iv: "75b34c997da959604089dfc6cd100393"
server_key_version: "encryptionkey@0"
I20260926 13:50:04.314975 13285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "3625c8f58abd4e1694562858b4a841d8"
format_stamp: "Formatted at 2026-09-26 13:50:04 on dist-test-slave-r34j"
server_key: "44b2e0723c4d7576831de0ed83071b4b"
server_key_iv: "75b34c997da959604089dfc6cd100393"
server_key_version: "encryptionkey@0"
I20260926 13:50:04.318560 13285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260926 13:50:04.321009 13300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:04.322249 13285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:04.322374 13285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "3625c8f58abd4e1694562858b4a841d8"
format_stamp: "Formatted at 2026-09-26 13:50:04 on dist-test-slave-r34j"
server_key: "44b2e0723c4d7576831de0ed83071b4b"
server_key_iv: "75b34c997da959604089dfc6cd100393"
server_key_version: "encryptionkey@0"
I20260926 13:50:04.322494 13285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:04.340895 13285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:04.344551 13285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:04.344810 13285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:04.346228 13285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:04.346307 13285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:04.346374 13285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:04.346418 13285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:04.356798 13285 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:43393
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430604, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:04.357854 13415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:43393 every 8 connection(s)
I20260926 13:50:04.357991 13285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:04.358564 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13285
I20260926 13:50:04.358676 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:04.358995 11187 external_mini_cluster.cc:1428] Setting key 6e98ca5816675f5ca937cac7a92d3161
I20260926 13:50:04.361888 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.347244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:34967 (local address 127.10.236.254:37681)
0926 13:50:04.347399 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.347403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.348554 (+ 1151us) server_negotiation.cc:407] Connection header received
0926 13:50:04.349794 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.349799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.349869 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.349978 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:04.351630 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.352590 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.354207 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.354614 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.354748 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.357785 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:04.357789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:04.357792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:04.357822 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:04.359803 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.360305 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.360308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.360310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.360397 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.360743 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.360746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.360748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.360940 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:04.361046 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.361578 (+ 532us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.361695 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:50:04.362718 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.347844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.10.236.195:34967)
0926 13:50:04.348409 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.348450 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.349525 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.350191 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.350198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.350669 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:04.351416 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.351438 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.352746 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.352749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.354074 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.354084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.354782 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.354786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.355005 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.355752 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:04.355768 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:04.357580 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:04.359908 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.359914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.359929 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.360184 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.360490 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.360494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.360496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.360641 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.361054 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:04.361063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:04.361342 (+ 279us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.361525 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.361854 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":139,"threads_started":1}
I20260926 13:50:04.364008 13416 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37681
I20260926 13:50:04.364166 13416 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:04.364694 13416 heartbeater.cc:511] Master 127.10.236.254:37681 requested a full tablet report, sending...
I20260926 13:50:04.365716 12933 ts_manager.cc:194] Registered new tserver with Master: 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393)
I20260926 13:50:04.366513 12933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:34967
I20260926 13:50:04.373286 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:50:04.374164 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.366203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:59247 (local address 127.10.236.254:37681)
0926 13:50:04.366313 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.366318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.366489 (+ 171us) server_negotiation.cc:407] Connection header received
0926 13:50:04.366641 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.366644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.366687 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.366755 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:04.367274 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.368089 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.368973 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.369179 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.369264 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.370526 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:04.370529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:04.370531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:04.370554 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:04.372490 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.372891 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.372895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.372897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.372992 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.373269 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.373272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.373273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.373512 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:04.373593 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.373923 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.374038 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:50:04.374184 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.366120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.10.236.195:59247)
0926 13:50:04.366409 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.366423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.366517 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.366803 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.366806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.367030 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:04.367156 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.367163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.368260 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.368262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.368852 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.368860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.369324 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.369327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.369410 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.369954 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:04.369968 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:04.370376 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:04.372582 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.372586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.372588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.372789 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.373081 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.373085 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.373087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.373172 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.373604 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:04.373608 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:04.373702 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.373819 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.373977 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":89,"threads_started":1}
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:04.401943 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.392707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37762 (local address 127.10.236.254:37681)
0926 13:50:04.392832 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.392859 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.392873 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:04.392915 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.392917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.392955 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.393048 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:04.394080 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.394622 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.395603 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.395823 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.395923 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.398512 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:04.398516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:04.398518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:04.398546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:04.400157 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.400625 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.400628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.400630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.400712 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.401067 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.401070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.401071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.401258 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:04.401381 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.401629 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.401728 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:50:04.405261 12933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:04.408502 12933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:04.424063 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.417408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:35027 (local address 127.0.0.1:51578)
0926 13:50:04.418007 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.418060 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.418178 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.419040 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.419048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.419074 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:04.419225 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.419232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.420909 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.420912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.422510 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.422521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.423417 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.423420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.423598 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.423650 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.423759 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.423891 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":63,"thread_start_us":88,"threads_started":1}
I20260926 13:50:04.424507 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.417593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.10.236.194:35027)
0926 13:50:04.418490 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.418502 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.418525 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:50:04.418601 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.418609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.418832 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.419383 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:04.419556 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.420780 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.422677 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.423286 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.423455 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.423646 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.424008 (+ 362us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.424122 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":149,"threads_started":1}
I20260926 13:50:04.425086 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.418156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.0.0.1:58434)
0926 13:50:04.418887 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.418899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.418969 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.419463 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.419466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.419511 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:04.419718 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.419723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.422093 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.422099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.423157 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.423168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.424695 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.424699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.424791 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.424815 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.424897 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.424991 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":43,"spinlock_wait_cycles":1408,"thread_start_us":89,"threads_started":1}
I20260926 13:50:04.425275 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.418545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.10.236.195:43393)
0926 13:50:04.418935 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.418946 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.418968 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:50:04.419108 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.419113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.419326 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.419457 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:04.421078 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.421978 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.423306 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.424123 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.424529 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.424840 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.425050 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.425136 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":89,"threads_started":1}
I20260926 13:50:04.425349 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.418181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.10.236.193:36421)
0926 13:50:04.418547 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.418555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.418793 (+ 238us) server_negotiation.cc:407] Connection header received
0926 13:50:04.419038 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.419045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.419206 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.419300 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:04.419840 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.420903 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.422640 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.423660 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.423790 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.423916 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.424141 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.424451 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":93,"threads_started":1}
I20260926 13:50:04.425598 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.417646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36421 (local address 127.0.0.1:54198)
0926 13:50:04.418696 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.418712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.418877 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.419463 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.419466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.419534 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:04.419716 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.419722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.421049 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.421053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.422518 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.422524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.423785 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.423788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.423886 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.423910 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.423990 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.424059 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":177,"spinlock_wait_cycles":86912,"thread_start_us":94,"threads_started":1}
I20260926 13:50:04.427522 13208 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc5173eb231460cb0e50670c4a2cefe (DEFAULT_TABLE table=test-table [id=716ffb9a39c947cab3bd54edab8c82fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:04.427716 13332 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc5173eb231460cb0e50670c4a2cefe (DEFAULT_TABLE table=test-table [id=716ffb9a39c947cab3bd54edab8c82fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:04.428665 13332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc5173eb231460cb0e50670c4a2cefe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.428659 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc5173eb231460cb0e50670c4a2cefe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.429731 13070 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc5173eb231460cb0e50670c4a2cefe (DEFAULT_TABLE table=test-table [id=716ffb9a39c947cab3bd54edab8c82fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:04.430675 13070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc5173eb231460cb0e50670c4a2cefe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.436630 13440 tablet_bootstrap.cc:492] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Bootstrap starting.
I20260926 13:50:04.437232 13441 tablet_bootstrap.cc:492] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Bootstrap starting.
I20260926 13:50:04.438647 13440 tablet_bootstrap.cc:654] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.439015 13441 tablet_bootstrap.cc:654] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.439459 13440 log.cc:824] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:04.439821 13441 log.cc:824] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:04.439841 13439 tablet_bootstrap.cc:492] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Bootstrap starting.
I20260926 13:50:04.441586 13440 tablet_bootstrap.cc:492] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: No bootstrap required, opened a new log
I20260926 13:50:04.441700 13441 tablet_bootstrap.cc:492] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: No bootstrap required, opened a new log
I20260926 13:50:04.441778 13440 ts_tablet_manager.cc:1397] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:50:04.441864 13439 tablet_bootstrap.cc:654] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.441879 13441 ts_tablet_manager.cc:1397] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:50:04.442687 13439 log.cc:824] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:04.444895 13439 tablet_bootstrap.cc:492] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: No bootstrap required, opened a new log
I20260926 13:50:04.445075 13439 ts_tablet_manager.cc:1397] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:50:04.445813 13441 raft_consensus.cc:348] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.445834 13440 raft_consensus.cc:348] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.446043 13441 raft_consensus.cc:374] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.446067 13440 raft_consensus.cc:374] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.446095 13441 raft_consensus.cc:729] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c58b4536044fcf8e5597ca32a26a10, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.446110 13440 raft_consensus.cc:729] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543647d1a5eb43f39199da4bb243585c, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.446527 13441 consensus_queue.cc:260] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.446533 13440 consensus_queue.cc:260] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.447108 13136 heartbeater.cc:503] Master 127.10.236.254:37681 was elected leader, sending a full tablet report...
I20260926 13:50:04.447458 13441 ts_tablet_manager.cc:1428] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 13:50:04.447516 13440 ts_tablet_manager.cc:1428] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:50:04.447875 13276 heartbeater.cc:503] Master 127.10.236.254:37681 was elected leader, sending a full tablet report...
I20260926 13:50:04.449939 13439 raft_consensus.cc:348] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.450209 13439 raft_consensus.cc:374] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.450264 13439 raft_consensus.cc:729] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.450724 13439 consensus_queue.cc:260] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.451577 13416 heartbeater.cc:503] Master 127.10.236.254:37681 was elected leader, sending a full tablet report...
I20260926 13:50:04.451905 13439 ts_tablet_manager.cc:1428] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:50:04.519055 13447 raft_consensus.cc:482] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:04.519315 13447 raft_consensus.cc:504] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.520478 13447 leader_election.cc:288] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027), 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421)
I20260926 13:50:04.525036 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.520791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36421 (local address 127.10.236.195:52789)
0926 13:50:04.520948 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.520972 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.521071 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.521426 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.521432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.521455 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:04.521582 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.521589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.522693 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.522696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.523658 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.523666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.524604 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.524607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.524710 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.524733 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.524809 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.524888 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 13:50:04.525031 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.520998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:52789 (local address 127.10.236.193:36421)
0926 13:50:04.521111 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.521115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.521141 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:50:04.521218 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.521221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.521273 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.521370 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:04.521730 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.522585 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.523779 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.524463 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.524609 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.524651 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.524860 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.524932 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:50:04.525691 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.520936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:40883 (local address 127.10.236.194:35027)
0926 13:50:04.521062 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.521067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.521447 (+ 380us) server_negotiation.cc:407] Connection header received
0926 13:50:04.521537 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.521543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.521593 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.521672 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:04.522422 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.523083 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.524422 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.525007 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.525124 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.525171 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.525513 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.525585 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:50:04.525713 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.520791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:35027 (local address 127.10.236.195:40883)
0926 13:50:04.521340 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.521355 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.521446 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.521727 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.521730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.521749 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:04.522283 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.522291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.523218 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.523220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.524297 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.524307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.525279 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.525284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.525374 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.525401 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.525479 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.525537 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":159,"thread_start_us":167,"threads_started":1}
I20260926 13:50:04.525738 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc5173eb231460cb0e50670c4a2cefe" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10" is_pre_election: true
I20260926 13:50:04.526182 13090 raft_consensus.cc:2454] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 0.
I20260926 13:50:04.526389 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc5173eb231460cb0e50670c4a2cefe" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c" is_pre_election: true
I20260926 13:50:04.526683 13230 raft_consensus.cc:2454] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 0.
I20260926 13:50:04.526760 13302 leader_election.cc:302] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 39c58b4536044fcf8e5597ca32a26a10; no voters:
I20260926 13:50:04.527122 13447 raft_consensus.cc:2781] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:04.527206 13447 raft_consensus.cc:482] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:04.527253 13447 raft_consensus.cc:3037] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.528345 13447 raft_consensus.cc:504] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.528692 13447 leader_election.cc:288] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 election: Requested vote from peers 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027), 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421)
I20260926 13:50:04.529013 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc5173eb231460cb0e50670c4a2cefe" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c"
I20260926 13:50:04.529145 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc5173eb231460cb0e50670c4a2cefe" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10"
I20260926 13:50:04.529140 13230 raft_consensus.cc:3037] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.529312 13090 raft_consensus.cc:3037] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.530264 13230 raft_consensus.cc:2454] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 1.
I20260926 13:50:04.530539 13090 raft_consensus.cc:2454] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 1.
I20260926 13:50:04.530571 13303 leader_election.cc:302] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:04.530736 13447 raft_consensus.cc:2781] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:04.530964 13447 raft_consensus.cc:686] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Running, Role: LEADER
I20260926 13:50:04.531291 13447 consensus_queue.cc:237] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.534700 12933 catalog_manager.cc:5495] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } health_report { overall_health: HEALTHY } } }
Sep 26 13:50:04 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430603, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:04.570420 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.561588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.10.236.254:37681)
0926 13:50:04.561726 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.561730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.561901 (+ 171us) server_negotiation.cc:407] Connection header received
0926 13:50:04.562002 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.562004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.562056 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.562144 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:04.562944 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.563474 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.564401 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.564644 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.564751 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.567072 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:04.567075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:04.567077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:04.567099 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:04.568571 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.569101 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.569105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.569106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.569218 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:04.569508 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:04.569511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:04.569512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:04.569689 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:04.569766 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.570133 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.570285 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":50}
I20260926 13:50:04.570420 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.561465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37681 (local address 127.0.0.1:37778)
0926 13:50:04.561793 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.561811 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.561930 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.562257 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.562260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.562710 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:04.562804 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.562810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.563665 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.563668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.564253 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.564286 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.564755 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.564758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.564837 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.565448 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:04.565459 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:04.566921 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:04.568684 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.568691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.568696 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.568982 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.569318 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:04.569320 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:04.569321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:04.569409 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:04.569829 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:04.569843 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:04.569965 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.570079 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.570218 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":161,"threads_started":1}
I20260926 13:50:04.574680 12933 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:37778:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 13:50:04.575476 12933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:04.581506 13208 tablet_service.cc:1467] Processing CreateTablet for tablet 81dd7310861545b2a533d40d0977f1ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=27e66242798541508999973619db81ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:04.581622 13332 tablet_service.cc:1467] Processing CreateTablet for tablet 81dd7310861545b2a533d40d0977f1ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=27e66242798541508999973619db81ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:04.582413 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dd7310861545b2a533d40d0977f1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.582499 13332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dd7310861545b2a533d40d0977f1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.585093 13070 tablet_service.cc:1467] Processing CreateTablet for tablet 81dd7310861545b2a533d40d0977f1ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=27e66242798541508999973619db81ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:04.585774 13070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dd7310861545b2a533d40d0977f1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.585834 13440 tablet_bootstrap.cc:492] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: Bootstrap starting.
I20260926 13:50:04.585987 13439 tablet_bootstrap.cc:492] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: Bootstrap starting.
I20260926 13:50:04.586769 13440 tablet_bootstrap.cc:654] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.587033 13439 tablet_bootstrap.cc:654] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.588874 13441 tablet_bootstrap.cc:492] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: Bootstrap starting.
I20260926 13:50:04.588973 13440 tablet_bootstrap.cc:492] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: No bootstrap required, opened a new log
I20260926 13:50:04.589066 13440 ts_tablet_manager.cc:1397] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:04.589078 13439 tablet_bootstrap.cc:492] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: No bootstrap required, opened a new log
I20260926 13:50:04.589155 13439 ts_tablet_manager.cc:1397] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:04.589581 13440 raft_consensus.cc:348] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.589609 13439 raft_consensus.cc:348] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.589712 13439 raft_consensus.cc:374] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.589712 13440 raft_consensus.cc:374] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.589756 13440 raft_consensus.cc:729] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543647d1a5eb43f39199da4bb243585c, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.589756 13439 raft_consensus.cc:729] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.589823 13441 tablet_bootstrap.cc:654] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.589900 13440 consensus_queue.cc:260] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.589896 13439 consensus_queue.cc:260] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.590204 13440 ts_tablet_manager.cc:1428] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 13:50:04.590154 13447 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.590286 13447 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:04.590673 13446 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.590811 13446 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:04.591022 13439 ts_tablet_manager.cc:1428] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:04.592048 13441 tablet_bootstrap.cc:492] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: No bootstrap required, opened a new log
I20260926 13:50:04.592175 13441 ts_tablet_manager.cc:1397] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:50:04.592694 13441 raft_consensus.cc:348] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.592820 13441 raft_consensus.cc:374] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.592859 13441 raft_consensus.cc:729] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c58b4536044fcf8e5597ca32a26a10, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.593031 13441 consensus_queue.cc:260] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.594558 13445 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.594705 13445 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:04.595081 13441 ts_tablet_manager.cc:1428] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 13:50:04.597164 13447 raft_consensus.cc:482] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:04.597261 13447 raft_consensus.cc:504] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.597733 13447 leader_election.cc:288] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421), 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027)
I20260926 13:50:04.598112 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd7310861545b2a533d40d0977f1ae" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10" is_pre_election: true
I20260926 13:50:04.598158 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd7310861545b2a533d40d0977f1ae" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c" is_pre_election: true
I20260926 13:50:04.598232 13090 raft_consensus.cc:2454] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 0.
I20260926 13:50:04.598320 13230 raft_consensus.cc:2454] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 0.
I20260926 13:50:04.598556 13302 leader_election.cc:302] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 39c58b4536044fcf8e5597ca32a26a10; no voters:
I20260926 13:50:04.598783 13447 raft_consensus.cc:2781] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:04.598832 13447 raft_consensus.cc:482] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:04.598865 13447 raft_consensus.cc:3037] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.599886 13447 raft_consensus.cc:504] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.600210 13447 leader_election.cc:288] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 election: Requested vote from peers 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421), 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027)
I20260926 13:50:04.600500 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd7310861545b2a533d40d0977f1ae" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10"
I20260926 13:50:04.600522 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd7310861545b2a533d40d0977f1ae" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c"
I20260926 13:50:04.600611 13090 raft_consensus.cc:3037] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.600642 13230 raft_consensus.cc:3037] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.601691 13230 raft_consensus.cc:2454] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 1.
I20260926 13:50:04.601722 13090 raft_consensus.cc:2454] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 1.
I20260926 13:50:04.601991 13303 leader_election.cc:302] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:04.602172 13447 raft_consensus.cc:2781] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:04.602277 13447 raft_consensus.cc:686] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Running, Role: LEADER
I20260926 13:50:04.602408 13447 consensus_queue.cc:237] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.603127 13449 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: TxnStatusTablet state changed. Reason: New leader 3625c8f58abd4e1694562858b4a841d8. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.603241 13449 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:04.603587 13461 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 13:50:04.603878 13461 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 13:50:04.604085 12933 catalog_manager.cc:5495] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } health_report { overall_health: HEALTHY } } }
W20260926 13:50:04.610106 13417 tablet.cc:2369] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:04.618058 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.613434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.0.0.1:58440)
0926 13:50:04.613772 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.613789 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.613902 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.614293 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.614297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.614305 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:04.614400 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.614405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.615614 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.615617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.616229 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.616237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.616878 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.616880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.616948 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.616991 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:04.617764 (+ 773us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:04.617778 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.617876 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.617947 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260926 13:50:04.618117 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.613602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.10.236.195:43393)
0926 13:50:04.613750 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.613758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.613865 (+ 107us) server_negotiation.cc:407] Connection header received
0926 13:50:04.614043 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.614047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.614116 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.614231 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:04.614518 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.615396 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.616353 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.616729 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.616861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.617131 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:04.617657 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:04.617784 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.617892 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.617959 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 13:50:04.624148 13230 raft_consensus.cc:1261] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Refusing update from remote peer 3625c8f58abd4e1694562858b4a841d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:04.624308 13090 raft_consensus.cc:1261] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Refusing update from remote peer 3625c8f58abd4e1694562858b4a841d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:04.624914 13447 consensus_queue.cc:1035] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:04.625069 13449 consensus_queue.cc:1035] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:04.628806 13445 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: TxnStatusTablet state changed. Reason: New leader 3625c8f58abd4e1694562858b4a841d8. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.628937 13445 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:04.628984 13446 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: TxnStatusTablet state changed. Reason: New leader 3625c8f58abd4e1694562858b4a841d8. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.629110 13446 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:04.632740 13447 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.632853 13447 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:04.635207 13449 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.635318 13449 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 3625c8f58abd4e1694562858b4a841d8: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:04.635563 13446 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
W20260926 13:50:04.635658 13277 tablet.cc:2369] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:04.635665 13446 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 543647d1a5eb43f39199da4bb243585c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:04.636624 13466 mvcc.cc:205] Tried to move back new op lower bound from 7333603756533383168 to 7333603756453019648. Current Snapshot: MvccSnapshot[applied={T|T < 7333603756533383168}]
I20260926 13:50:04.637647 13445 tablet_replica.cc:439] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } } }
I20260926 13:50:04.637754 13445 tablet_replica.cc:442] T 81dd7310861545b2a533d40d0977f1ae P 39c58b4536044fcf8e5597ca32a26a10: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 13:50:04.640148 13137 tablet.cc:2369] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:04.648038 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.644317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.10.236.195:43393)
0926 13:50:04.644458 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.644462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.644478 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:04.644638 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.644642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.644690 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.644778 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:04.645175 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.645954 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.646847 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.647190 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.647289 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.647594 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:04.647676 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:04.647750 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.647847 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.647903 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:04.660391 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.656107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.10.236.195:45869)
0926 13:50:04.656338 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:04.656358 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:04.656467 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:04.656749 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:04.656752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:04.656758 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:04.656899 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.656904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.657894 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.657897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.658900 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:04.658910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.659556 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.659559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:04.659673 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.659764 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:04.660103 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:04.660110 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:50:04.660178 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:50:04.660239 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 13:50:04.660416 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:04.656309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45869 (local address 127.10.236.195:43393)
0926 13:50:04.656434 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:04.656439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:04.656455 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:04.656564 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:04.656567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:04.656615 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:04.656694 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:04.657006 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.657762 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.659020 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:04.659389 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:04.659510 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:04.659922 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:04.659999 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:04.660072 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:04.660208 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 13:50:04.660255 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:50:04.667552 13090 raft_consensus.cc:1261] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Refusing update from remote peer 3625c8f58abd4e1694562858b4a841d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:04.667622 13230 raft_consensus.cc:1261] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Refusing update from remote peer 3625c8f58abd4e1694562858b4a841d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:04.667966 13449 consensus_queue.cc:1035] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:04.668257 13449 consensus_queue.cc:1035] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:04.709499 12932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:04.709664 12932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:04.712265 12932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Sending DeleteTablet for 3 replicas of tablet 3bc5173eb231460cb0e50670c4a2cefe
I20260926 13:50:04.713073 13070 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc5173eb231460cb0e50670c4a2cefe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:04 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:54198
I20260926 13:50:04.713073 13208 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc5173eb231460cb0e50670c4a2cefe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:04 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51578
I20260926 13:50:04.713155 13350 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc5173eb231460cb0e50670c4a2cefe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:04 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260926 13:50:04.713670 13481 tablet_replica.cc:330] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: stopping tablet replica
I20260926 13:50:04.714005 13481 raft_consensus.cc:2229] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:04.714231 13481 raft_consensus.cc:2258] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:04.714666 13483 tablet_replica.cc:330] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: stopping tablet replica
I20260926 13:50:04.715016 13483 raft_consensus.cc:2229] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:04.715166 12932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 13:50:04.715298 13483 raft_consensus.cc:2258] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 13:50:04.715636 12932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:04.716015 13482 tablet_replica.cc:330] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: stopping tablet replica
I20260926 13:50:04.716224 13481 ts_tablet_manager.cc:1905] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:04.716418 13482 raft_consensus.cc:2229] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:04.716797 13482 raft_consensus.cc:2258] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:04.718268 13482 ts_tablet_manager.cc:1905] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:04.721081 13483 ts_tablet_manager.cc:1905] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:04.721340 13482 ts_tablet_manager.cc:1918] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:04.721658 13208 tablet_service.cc:1467] Processing CreateTablet for tablet 08667cda7ea547e88b8a6cee3d116afa (DEFAULT_TABLE table=test-table [id=4e83865d56d7453fae62b1102f6bd8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:04.721758 13070 tablet_service.cc:1467] Processing CreateTablet for tablet 08667cda7ea547e88b8a6cee3d116afa (DEFAULT_TABLE table=test-table [id=4e83865d56d7453fae62b1102f6bd8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:04.721971 13482 log.cc:1196] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/3bc5173eb231460cb0e50670c4a2cefe
I20260926 13:50:04.721992 13070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08667cda7ea547e88b8a6cee3d116afa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.722409 13482 ts_tablet_manager.cc:1939] T 3bc5173eb231460cb0e50670c4a2cefe P 3625c8f58abd4e1694562858b4a841d8: Deleting consensus metadata
I20260926 13:50:04.723721 12919 catalog_manager.cc:4841] TS 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393): tablet 3bc5173eb231460cb0e50670c4a2cefe (table test-table [id=716ffb9a39c947cab3bd54edab8c82fa]) successfully deleted
I20260926 13:50:04.724265 13350 tablet_service.cc:1467] Processing CreateTablet for tablet 08667cda7ea547e88b8a6cee3d116afa (DEFAULT_TABLE table=test-table [id=4e83865d56d7453fae62b1102f6bd8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:04.724463 13441 tablet_bootstrap.cc:492] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Bootstrap starting.
I20260926 13:50:04.724509 13350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08667cda7ea547e88b8a6cee3d116afa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.725414 13441 tablet_bootstrap.cc:654] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.725502 13483 ts_tablet_manager.cc:1918] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:04.725586 13483 log.cc:1196] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/3bc5173eb231460cb0e50670c4a2cefe
I20260926 13:50:04.725891 13483 ts_tablet_manager.cc:1939] T 3bc5173eb231460cb0e50670c4a2cefe P 39c58b4536044fcf8e5597ca32a26a10: Deleting consensus metadata
I20260926 13:50:04.726652 13441 tablet_bootstrap.cc:492] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: No bootstrap required, opened a new log
I20260926 13:50:04.726698 13439 tablet_bootstrap.cc:492] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Bootstrap starting.
I20260926 13:50:04.726744 13441 ts_tablet_manager.cc:1397] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:04.727201 13441 raft_consensus.cc:348] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.727305 13441 raft_consensus.cc:374] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.727336 13441 raft_consensus.cc:729] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c58b4536044fcf8e5597ca32a26a10, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.727455 13441 consensus_queue.cc:260] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.727603 13439 tablet_bootstrap.cc:654] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.727801 13441 ts_tablet_manager.cc:1428] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 13:50:04.727838 12918 catalog_manager.cc:4841] TS 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421): tablet 3bc5173eb231460cb0e50670c4a2cefe (table test-table [id=716ffb9a39c947cab3bd54edab8c82fa]) successfully deleted
I20260926 13:50:04.728632 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08667cda7ea547e88b8a6cee3d116afa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:04.728633 13481 ts_tablet_manager.cc:1918] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:04.728900 13481 log.cc:1196] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/3bc5173eb231460cb0e50670c4a2cefe
I20260926 13:50:04.729324 13481 ts_tablet_manager.cc:1939] T 3bc5173eb231460cb0e50670c4a2cefe P 543647d1a5eb43f39199da4bb243585c: Deleting consensus metadata
I20260926 13:50:04.730548 12919 catalog_manager.cc:4841] TS 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027): tablet 3bc5173eb231460cb0e50670c4a2cefe (table test-table [id=716ffb9a39c947cab3bd54edab8c82fa]) successfully deleted
I20260926 13:50:04.731235 13440 tablet_bootstrap.cc:492] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Bootstrap starting.
I20260926 13:50:04.731390 13439 tablet_bootstrap.cc:492] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: No bootstrap required, opened a new log
I20260926 13:50:04.731459 13439 ts_tablet_manager.cc:1397] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260926 13:50:04.731981 13439 raft_consensus.cc:348] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.732108 13439 raft_consensus.cc:374] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.732126 13440 tablet_bootstrap.cc:654] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:04.732146 13439 raft_consensus.cc:729] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.732286 13439 consensus_queue.cc:260] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.732565 13439 ts_tablet_manager.cc:1428] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:50:04.733500 13440 tablet_bootstrap.cc:492] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: No bootstrap required, opened a new log
I20260926 13:50:04.733613 13440 ts_tablet_manager.cc:1397] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:04.734012 13440 raft_consensus.cc:348] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.734123 13440 raft_consensus.cc:374] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:04.734181 13440 raft_consensus.cc:729] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543647d1a5eb43f39199da4bb243585c, State: Initialized, Role: FOLLOWER
I20260926 13:50:04.734331 13440 consensus_queue.cc:260] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.734604 13440 ts_tablet_manager.cc:1428] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:50:04.832967 13475 raft_consensus.cc:482] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:04.833163 13475 raft_consensus.cc:504] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.833694 13475 leader_election.cc:288] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421), 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027)
I20260926 13:50:04.834203 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08667cda7ea547e88b8a6cee3d116afa" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c" is_pre_election: true
I20260926 13:50:04.834214 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08667cda7ea547e88b8a6cee3d116afa" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10" is_pre_election: true
I20260926 13:50:04.834368 13230 raft_consensus.cc:2454] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 0.
I20260926 13:50:04.834368 13090 raft_consensus.cc:2454] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 0.
I20260926 13:50:04.834689 13303 leader_election.cc:302] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:04.835038 13475 raft_consensus.cc:2781] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:04.835117 13475 raft_consensus.cc:482] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:04.835156 13475 raft_consensus.cc:3037] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.836611 13475 raft_consensus.cc:504] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.837009 13475 leader_election.cc:288] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 election: Requested vote from peers 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421), 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027)
I20260926 13:50:04.837491 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08667cda7ea547e88b8a6cee3d116afa" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c"
I20260926 13:50:04.837487 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08667cda7ea547e88b8a6cee3d116afa" candidate_uuid: "3625c8f58abd4e1694562858b4a841d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10"
I20260926 13:50:04.837617 13230 raft_consensus.cc:3037] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.837648 13090 raft_consensus.cc:3037] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:04.838672 13090 raft_consensus.cc:2454] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 1.
I20260926 13:50:04.838680 13230 raft_consensus.cc:2454] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3625c8f58abd4e1694562858b4a841d8 in term 1.
I20260926 13:50:04.839046 13302 leader_election.cc:302] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 39c58b4536044fcf8e5597ca32a26a10; no voters:
I20260926 13:50:04.839246 13475 raft_consensus.cc:2781] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:04.839368 13475 raft_consensus.cc:686] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Running, Role: LEADER
I20260926 13:50:04.839563 13475 consensus_queue.cc:237] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:04.841533 12932 catalog_manager.cc:5495] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "3625c8f58abd4e1694562858b4a841d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } health_report { overall_health: HEALTHY } } }
I20260926 13:50:04.861994 13090 raft_consensus.cc:1261] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Refusing update from remote peer 3625c8f58abd4e1694562858b4a841d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:04.862466 13230 raft_consensus.cc:1261] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Refusing update from remote peer 3625c8f58abd4e1694562858b4a841d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:04.862421 13475 consensus_queue.cc:1035] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:04.862811 13478 consensus_queue.cc:1035] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:04.866097 13464 mvcc.cc:205] Tried to move back new op lower bound from 7333603757511884800 to 7333603757424832512. Current Snapshot: MvccSnapshot[applied={T|T < 7333603757511884800}]
I20260926 13:50:04.869191 13465 mvcc.cc:205] Tried to move back new op lower bound from 7333603757511884800 to 7333603757424832512. Current Snapshot: MvccSnapshot[applied={T|T < 7333603757511884800}]
W20260926 13:50:04.878332 13464 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 13:50:06.446539 12933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:06.446708 12933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:06.449707 12933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Sending DeleteTablet for 3 replicas of tablet 08667cda7ea547e88b8a6cee3d116afa
I20260926 13:50:06.450297 13070 tablet_service.cc:1514] Processing DeleteTablet for tablet 08667cda7ea547e88b8a6cee3d116afa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:54198
I20260926 13:50:06.450592 13350 tablet_service.cc:1514] Processing DeleteTablet for tablet 08667cda7ea547e88b8a6cee3d116afa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260926 13:50:06.450614 13208 tablet_service.cc:1514] Processing DeleteTablet for tablet 08667cda7ea547e88b8a6cee3d116afa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51578
I20260926 13:50:06.450927 13511 tablet_replica.cc:330] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: stopping tablet replica
I20260926 13:50:06.450934 13513 tablet_replica.cc:330] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: stopping tablet replica
I20260926 13:50:06.451049 13513 raft_consensus.cc:2229] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:06.451049 13511 raft_consensus.cc:2229] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:06.451164 13511 raft_consensus.cc:2258] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:06.451237 13513 raft_consensus.cc:2258] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:06.451758 13512 tablet_replica.cc:330] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: stopping tablet replica
I20260926 13:50:06.451838 13511 ts_tablet_manager.cc:1905] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:06.451895 13512 raft_consensus.cc:2229] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:06.452018 13512 raft_consensus.cc:2258] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:06.452126 13513 ts_tablet_manager.cc:1905] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:06.452639 13512 ts_tablet_manager.cc:1905] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:06.454855 13513 ts_tablet_manager.cc:1918] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:50:06.454941 13513 log.cc:1196] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/08667cda7ea547e88b8a6cee3d116afa
I20260926 13:50:06.454937 13511 ts_tablet_manager.cc:1918] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:50:06.455011 13511 log.cc:1196] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/08667cda7ea547e88b8a6cee3d116afa
I20260926 13:50:06.455147 13512 ts_tablet_manager.cc:1918] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:50:06.455243 13512 log.cc:1196] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/08667cda7ea547e88b8a6cee3d116afa
I20260926 13:50:06.455259 13513 ts_tablet_manager.cc:1939] T 08667cda7ea547e88b8a6cee3d116afa P 3625c8f58abd4e1694562858b4a841d8: Deleting consensus metadata
I20260926 13:50:06.455292 13511 ts_tablet_manager.cc:1939] T 08667cda7ea547e88b8a6cee3d116afa P 39c58b4536044fcf8e5597ca32a26a10: Deleting consensus metadata
I20260926 13:50:06.455560 13512 ts_tablet_manager.cc:1939] T 08667cda7ea547e88b8a6cee3d116afa P 543647d1a5eb43f39199da4bb243585c: Deleting consensus metadata
I20260926 13:50:06.456022 12919 catalog_manager.cc:4841] TS 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393): tablet 08667cda7ea547e88b8a6cee3d116afa (table test-table [id=4e83865d56d7453fae62b1102f6bd8c6]) successfully deleted
I20260926 13:50:06.456022 12918 catalog_manager.cc:4841] TS 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421): tablet 08667cda7ea547e88b8a6cee3d116afa (table test-table [id=4e83865d56d7453fae62b1102f6bd8c6]) successfully deleted
I20260926 13:50:06.456238 12919 catalog_manager.cc:4841] TS 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027): tablet 08667cda7ea547e88b8a6cee3d116afa (table test-table [id=4e83865d56d7453fae62b1102f6bd8c6]) successfully deleted
Sep 26 13:50:06 dist-test-slave-r34j krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430606, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:50:06 dist-test-slave-r34j krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430606, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:06.487653 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.477133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.10.236.254:37681)
0926 13:50:06.477440 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.477447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.477462 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:06.477570 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.477572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.477640 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.477743 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:06.478509 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.479138 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.480406 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.480645 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.480755 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.483955 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:06.483964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:06.483967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:06.483991 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:06.485698 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:06.486114 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:06.486118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:06.486120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:06.486217 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:06.486496 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:06.486499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:06.486501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:06.486688 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:06.486847 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.487098 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.487226 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":106,"threads_started":1}
I20260926 13:50:06.490483 12933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:06.490983 12933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:06.495816 13070 tablet_service.cc:1467] Processing CreateTablet for tablet a6913ec7964b48c8b699ecd2b0c35ad9 (DEFAULT_TABLE table=test-table [id=d418839a69c7413cb51f3fba0bff85f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:06.495862 13350 tablet_service.cc:1467] Processing CreateTablet for tablet a6913ec7964b48c8b699ecd2b0c35ad9 (DEFAULT_TABLE table=test-table [id=d418839a69c7413cb51f3fba0bff85f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:06.496145 13350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6913ec7964b48c8b699ecd2b0c35ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:06.496145 13070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6913ec7964b48c8b699ecd2b0c35ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:06.498667 13522 tablet_bootstrap.cc:492] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Bootstrap starting.
I20260926 13:50:06.498845 13208 tablet_service.cc:1467] Processing CreateTablet for tablet a6913ec7964b48c8b699ecd2b0c35ad9 (DEFAULT_TABLE table=test-table [id=d418839a69c7413cb51f3fba0bff85f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:06.499135 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6913ec7964b48c8b699ecd2b0c35ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:06.499755 13522 tablet_bootstrap.cc:654] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:06.500633 13523 tablet_bootstrap.cc:492] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Bootstrap starting.
I20260926 13:50:06.501232 13525 tablet_bootstrap.cc:492] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Bootstrap starting.
I20260926 13:50:06.501748 13523 tablet_bootstrap.cc:654] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:06.501974 13525 tablet_bootstrap.cc:654] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:06.502780 13522 tablet_bootstrap.cc:492] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: No bootstrap required, opened a new log
I20260926 13:50:06.502914 13522 ts_tablet_manager.cc:1397] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 13:50:06.502971 13523 tablet_bootstrap.cc:492] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: No bootstrap required, opened a new log
I20260926 13:50:06.503064 13523 ts_tablet_manager.cc:1397] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:06.503111 13525 tablet_bootstrap.cc:492] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: No bootstrap required, opened a new log
I20260926 13:50:06.503194 13525 ts_tablet_manager.cc:1397] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:06.503429 13522 raft_consensus.cc:348] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.503607 13522 raft_consensus.cc:374] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:06.503655 13522 raft_consensus.cc:729] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Initialized, Role: FOLLOWER
I20260926 13:50:06.503692 13525 raft_consensus.cc:348] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.503819 13525 raft_consensus.cc:374] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:06.503773 13523 raft_consensus.cc:348] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.503863 13525 raft_consensus.cc:729] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543647d1a5eb43f39199da4bb243585c, State: Initialized, Role: FOLLOWER
I20260926 13:50:06.503878 13523 raft_consensus.cc:374] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:06.503821 13522 consensus_queue.cc:260] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.503914 13523 raft_consensus.cc:729] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c58b4536044fcf8e5597ca32a26a10, State: Initialized, Role: FOLLOWER
I20260926 13:50:06.504132 13522 ts_tablet_manager.cc:1428] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:06.504133 13525 consensus_queue.cc:260] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.504590 13525 ts_tablet_manager.cc:1428] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:06.504592 13523 consensus_queue.cc:260] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.505041 13523 ts_tablet_manager.cc:1428] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:06.601565 13529 raft_consensus.cc:482] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:06.601706 13529 raft_consensus.cc:504] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.602854 13529 leader_election.cc:288] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027), 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393)
I20260926 13:50:06.607672 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.603913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:43523 (local address 127.10.236.195:43393)
0926 13:50:06.604244 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.604253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.604268 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:06.604321 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.604324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.604376 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.604482 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:06.604828 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.605442 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.606517 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.606998 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.607139 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.607200 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.607396 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.607509 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":147,"threads_started":1}
I20260926 13:50:06.607688 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.603771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.10.236.193:43523)
0926 13:50:06.604124 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.604138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.604214 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.604550 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.604554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.604573 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:06.604693 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.604700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.605570 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.605572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.606388 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.606397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.607131 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.607137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.607262 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.607292 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.607353 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.607431 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260926 13:50:06.607700 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.603199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:35027 (local address 127.10.236.193:44697)
0926 13:50:06.603647 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.603661 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.603853 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.604228 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.604232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.604261 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:06.604415 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.604426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.605328 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.605332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.606339 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.606349 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.607125 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.607129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.607308 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.607327 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.607432 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.607528 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":57,"thread_start_us":158,"threads_started":1}
I20260926 13:50:06.608063 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.603377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44697 (local address 127.10.236.194:35027)
0926 13:50:06.603748 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.603759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.603778 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:50:06.604003 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.604006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.604059 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.604182 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:06.604552 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.605202 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.606565 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.607001 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.607150 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.607202 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.607463 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.607583 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":144,"threads_started":1}
I20260926 13:50:06.608502 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6913ec7964b48c8b699ecd2b0c35ad9" candidate_uuid: "39c58b4536044fcf8e5597ca32a26a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c" is_pre_election: true
I20260926 13:50:06.608677 13230 raft_consensus.cc:2454] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c58b4536044fcf8e5597ca32a26a10 in term 0.
I20260926 13:50:06.608618 13353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6913ec7964b48c8b699ecd2b0c35ad9" candidate_uuid: "39c58b4536044fcf8e5597ca32a26a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3625c8f58abd4e1694562858b4a841d8" is_pre_election: true
I20260926 13:50:06.608901 13353 raft_consensus.cc:2454] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39c58b4536044fcf8e5597ca32a26a10 in term 0.
I20260926 13:50:06.609148 13023 leader_election.cc:302] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c58b4536044fcf8e5597ca32a26a10, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:06.609440 13529 raft_consensus.cc:2781] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:06.609499 13529 raft_consensus.cc:482] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:06.609527 13529 raft_consensus.cc:3037] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:06.610569 13529 raft_consensus.cc:504] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.610924 13529 leader_election.cc:288] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [CANDIDATE]: Term 1 election: Requested vote from peers 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027), 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393)
I20260926 13:50:06.611361 13230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6913ec7964b48c8b699ecd2b0c35ad9" candidate_uuid: "39c58b4536044fcf8e5597ca32a26a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "543647d1a5eb43f39199da4bb243585c"
I20260926 13:50:06.611382 13353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6913ec7964b48c8b699ecd2b0c35ad9" candidate_uuid: "39c58b4536044fcf8e5597ca32a26a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3625c8f58abd4e1694562858b4a841d8"
I20260926 13:50:06.611466 13230 raft_consensus.cc:3037] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:06.611536 13353 raft_consensus.cc:3037] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:06.612401 13230 raft_consensus.cc:2454] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c58b4536044fcf8e5597ca32a26a10 in term 1.
I20260926 13:50:06.612664 13353 raft_consensus.cc:2454] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39c58b4536044fcf8e5597ca32a26a10 in term 1.
I20260926 13:50:06.612727 13023 leader_election.cc:302] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c58b4536044fcf8e5597ca32a26a10, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:06.612915 13529 raft_consensus.cc:2781] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:06.613137 13529 raft_consensus.cc:686] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 1 LEADER]: Becoming Leader. State: Replica: 39c58b4536044fcf8e5597ca32a26a10, State: Running, Role: LEADER
I20260926 13:50:06.613405 13529 consensus_queue.cc:237] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.615901 12933 catalog_manager.cc:5495] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193). New cstate: current_term: 1 leader_uuid: "39c58b4536044fcf8e5597ca32a26a10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:06.633294 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.629060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.10.236.193:36421)
0926 13:50:06.629339 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.629347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.629361 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:06.629442 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.629444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.629504 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.629628 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:06.629935 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.630785 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.631714 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.632093 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.632192 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.632431 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.632941 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.633029 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.633124 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.633169 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":111,"threads_started":1}
I20260926 13:50:06.644299 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.640581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:43485 (local address 127.10.236.195:43393)
0926 13:50:06.640706 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.640709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.640917 (+ 208us) server_negotiation.cc:407] Connection header received
0926 13:50:06.641040 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.641042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.641086 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.641150 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:06.641420 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.641950 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.642981 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.643259 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.643394 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.643774 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.643869 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.643930 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.644062 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.644146 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:50:06.644320 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.640499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.10.236.193:43485)
0926 13:50:06.640825 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.640845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.640935 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.641193 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.641195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.641202 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:06.641296 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.641300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.642091 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.642095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.642871 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.642881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.643446 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.643449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.643584 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.643630 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.643959 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.643967 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.644044 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.644109 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":120,"threads_started":1}
I20260926 13:50:06.650530 13230 raft_consensus.cc:1261] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Refusing update from remote peer 39c58b4536044fcf8e5597ca32a26a10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:06.650708 13353 raft_consensus.cc:1261] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Refusing update from remote peer 39c58b4536044fcf8e5597ca32a26a10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:06.651090 13529 consensus_queue.cc:1035] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:06.651273 13534 consensus_queue.cc:1035] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:06.670617 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.666579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36421 (local address 127.10.236.195:41377)
0926 13:50:06.666914 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.666931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.667040 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.667327 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.667329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.667341 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:06.667454 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.667459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.668202 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.668204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.668985 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.668994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.669589 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.669593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.669691 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.669732 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.670165 (+ 433us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.670205 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.670380 (+ 175us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.670470 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":128,"threads_started":1}
I20260926 13:50:06.670612 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.666646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:41377 (local address 127.10.236.193:36421)
0926 13:50:06.666781 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.666786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.667026 (+ 240us) server_negotiation.cc:407] Connection header received
0926 13:50:06.667138 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.667143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.667219 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.667297 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:06.667591 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.668096 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.669115 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.669390 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.669536 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.669939 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.670038 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.670127 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.670399 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.670461 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:50:06.694530 12933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:06.694680 12933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:06.696846 12933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Sending DeleteTablet for 3 replicas of tablet a6913ec7964b48c8b699ecd2b0c35ad9
I20260926 13:50:06.697398 13070 tablet_service.cc:1514] Processing DeleteTablet for tablet a6913ec7964b48c8b699ecd2b0c35ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:54198
I20260926 13:50:06.697510 13350 tablet_service.cc:1514] Processing DeleteTablet for tablet a6913ec7964b48c8b699ecd2b0c35ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260926 13:50:06.697556 13208 tablet_service.cc:1514] Processing DeleteTablet for tablet a6913ec7964b48c8b699ecd2b0c35ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51578
I20260926 13:50:06.697592 13511 tablet_replica.cc:330] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: stopping tablet replica
I20260926 13:50:06.697692 13513 tablet_replica.cc:330] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: stopping tablet replica
I20260926 13:50:06.697723 13511 raft_consensus.cc:2229] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:06.697796 13513 raft_consensus.cc:2229] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:06.697896 13513 raft_consensus.cc:2258] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:06.698002 13511 raft_consensus.cc:2258] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:06.698170 13512 tablet_replica.cc:330] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: stopping tablet replica
I20260926 13:50:06.698285 13512 raft_consensus.cc:2229] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:06.698393 13512 raft_consensus.cc:2258] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:06.698536 12933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 13:50:06.698746 13513 ts_tablet_manager.cc:1905] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:50:06.698993 12933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:06.699198 13512 ts_tablet_manager.cc:1905] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:06.699416 13511 ts_tablet_manager.cc:1905] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:06.701472 13513 ts_tablet_manager.cc:1918] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:06.701545 13513 log.cc:1196] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/a6913ec7964b48c8b699ecd2b0c35ad9
I20260926 13:50:06.701749 13512 ts_tablet_manager.cc:1918] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:06.701798 13513 ts_tablet_manager.cc:1939] T a6913ec7964b48c8b699ecd2b0c35ad9 P 3625c8f58abd4e1694562858b4a841d8: Deleting consensus metadata
I20260926 13:50:06.701817 13512 log.cc:1196] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/a6913ec7964b48c8b699ecd2b0c35ad9
I20260926 13:50:06.702098 13512 ts_tablet_manager.cc:1939] T a6913ec7964b48c8b699ecd2b0c35ad9 P 543647d1a5eb43f39199da4bb243585c: Deleting consensus metadata
I20260926 13:50:06.702087 13511 ts_tablet_manager.cc:1918] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:06.702174 13511 log.cc:1196] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/a6913ec7964b48c8b699ecd2b0c35ad9
I20260926 13:50:06.702451 13511 ts_tablet_manager.cc:1939] T a6913ec7964b48c8b699ecd2b0c35ad9 P 39c58b4536044fcf8e5597ca32a26a10: Deleting consensus metadata
I20260926 13:50:06.703238 12919 catalog_manager.cc:4841] TS 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027): tablet a6913ec7964b48c8b699ecd2b0c35ad9 (table test-table [id=d418839a69c7413cb51f3fba0bff85f6]) successfully deleted
I20260926 13:50:06.703970 13070 tablet_service.cc:1467] Processing CreateTablet for tablet 635ea9923acd4300a508f4b46857826d (DEFAULT_TABLE table=test-table [id=f86845bff0114203abcc4c08f594e61a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:06.704172 12919 catalog_manager.cc:4841] TS 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393): tablet a6913ec7964b48c8b699ecd2b0c35ad9 (table test-table [id=d418839a69c7413cb51f3fba0bff85f6]) successfully deleted
I20260926 13:50:06.704238 13070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635ea9923acd4300a508f4b46857826d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:06.704282 12918 catalog_manager.cc:4841] TS 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421): tablet a6913ec7964b48c8b699ecd2b0c35ad9 (table test-table [id=d418839a69c7413cb51f3fba0bff85f6]) successfully deleted
I20260926 13:50:06.704711 13350 tablet_service.cc:1467] Processing CreateTablet for tablet 635ea9923acd4300a508f4b46857826d (DEFAULT_TABLE table=test-table [id=f86845bff0114203abcc4c08f594e61a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:06.704947 13350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635ea9923acd4300a508f4b46857826d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:06.706329 13523 tablet_bootstrap.cc:492] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10: Bootstrap starting.
I20260926 13:50:06.706507 13208 tablet_service.cc:1467] Processing CreateTablet for tablet 635ea9923acd4300a508f4b46857826d (DEFAULT_TABLE table=test-table [id=f86845bff0114203abcc4c08f594e61a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:06.706732 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635ea9923acd4300a508f4b46857826d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:06.707175 13522 tablet_bootstrap.cc:492] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Bootstrap starting.
I20260926 13:50:06.707226 13523 tablet_bootstrap.cc:654] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:06.707988 13522 tablet_bootstrap.cc:654] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:06.708510 13523 tablet_bootstrap.cc:492] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10: No bootstrap required, opened a new log
I20260926 13:50:06.708587 13523 ts_tablet_manager.cc:1397] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:06.708922 13525 tablet_bootstrap.cc:492] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Bootstrap starting.
I20260926 13:50:06.708992 13523 raft_consensus.cc:348] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.709087 13523 raft_consensus.cc:374] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:06.709117 13523 raft_consensus.cc:729] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c58b4536044fcf8e5597ca32a26a10, State: Initialized, Role: FOLLOWER
I20260926 13:50:06.709244 13523 consensus_queue.cc:260] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.709542 13523 ts_tablet_manager.cc:1428] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:06.709741 13522 tablet_bootstrap.cc:492] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: No bootstrap required, opened a new log
I20260926 13:50:06.709824 13522 ts_tablet_manager.cc:1397] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 13:50:06.709846 13525 tablet_bootstrap.cc:654] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:06.710206 13522 raft_consensus.cc:348] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.710319 13522 raft_consensus.cc:374] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:06.710367 13522 raft_consensus.cc:729] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3625c8f58abd4e1694562858b4a841d8, State: Initialized, Role: FOLLOWER
I20260926 13:50:06.710495 13522 consensus_queue.cc:260] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.710785 13522 ts_tablet_manager.cc:1428] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:06.711103 13525 tablet_bootstrap.cc:492] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: No bootstrap required, opened a new log
I20260926 13:50:06.711210 13525 ts_tablet_manager.cc:1397] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:06.711745 13525 raft_consensus.cc:348] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.711853 13525 raft_consensus.cc:374] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:06.711892 13525 raft_consensus.cc:729] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543647d1a5eb43f39199da4bb243585c, State: Initialized, Role: FOLLOWER
I20260926 13:50:06.712036 13525 consensus_queue.cc:260] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.712308 13525 ts_tablet_manager.cc:1428] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:06.914919 13528 raft_consensus.cc:482] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:06.915124 13528 raft_consensus.cc:504] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.916271 13528 leader_election.cc:288] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421), 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393)
I20260926 13:50:06.920262 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.916692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:55609 (local address 127.10.236.193:36421)
0926 13:50:06.916828 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.916833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.917080 (+ 247us) server_negotiation.cc:407] Connection header received
0926 13:50:06.917228 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.917232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.917284 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.917370 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:06.917631 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.918128 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.919201 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.919694 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.919824 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.919883 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.920060 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.920131 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:50:06.920262 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.916768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:39659 (local address 127.10.236.195:43393)
0926 13:50:06.916889 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.916893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.917029 (+ 136us) server_negotiation.cc:407] Connection header received
0926 13:50:06.917182 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.917190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.917232 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.917311 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:06.917577 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.918091 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.919219 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.919683 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.919809 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.919863 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.920058 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.920131 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:50:06.920285 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.916582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.10.236.194:39659)
0926 13:50:06.916906 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.916921 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.917086 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.917332 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.917335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.917349 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:06.917472 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.917478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.918210 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.918213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.919095 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.919103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.919804 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.919807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.919926 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.919951 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.920046 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.920144 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":62,"thread_start_us":74,"threads_started":1}
I20260926 13:50:06.920293 13551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.916582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36421 (local address 127.10.236.194:55609)
0926 13:50:06.916992 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.917004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.917109 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.917383 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.917386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.917402 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:06.917523 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.917528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.918254 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.918258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.919078 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.919086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.919811 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.919814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.919936 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.919956 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.920046 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.920130 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":62,"thread_start_us":101,"threads_started":1}
I20260926 13:50:06.920735 13353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635ea9923acd4300a508f4b46857826d" candidate_uuid: "543647d1a5eb43f39199da4bb243585c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3625c8f58abd4e1694562858b4a841d8" is_pre_election: true
I20260926 13:50:06.920897 13353 raft_consensus.cc:2454] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543647d1a5eb43f39199da4bb243585c in term 0.
I20260926 13:50:06.921036 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635ea9923acd4300a508f4b46857826d" candidate_uuid: "543647d1a5eb43f39199da4bb243585c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10" is_pre_election: true
I20260926 13:50:06.921165 13089 raft_consensus.cc:2454] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543647d1a5eb43f39199da4bb243585c in term 0.
I20260926 13:50:06.921267 13163 leader_election.cc:302] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:06.921542 13528 raft_consensus.cc:2781] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:06.921607 13528 raft_consensus.cc:482] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:06.921635 13528 raft_consensus.cc:3037] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:06.922619 13528 raft_consensus.cc:504] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.922984 13528 leader_election.cc:288] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [CANDIDATE]: Term 1 election: Requested vote from peers 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421), 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393)
I20260926 13:50:06.923326 13353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635ea9923acd4300a508f4b46857826d" candidate_uuid: "543647d1a5eb43f39199da4bb243585c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3625c8f58abd4e1694562858b4a841d8"
I20260926 13:50:06.923377 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635ea9923acd4300a508f4b46857826d" candidate_uuid: "543647d1a5eb43f39199da4bb243585c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c58b4536044fcf8e5597ca32a26a10"
I20260926 13:50:06.923453 13353 raft_consensus.cc:3037] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:06.923547 13089 raft_consensus.cc:3037] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:06.924396 13353 raft_consensus.cc:2454] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543647d1a5eb43f39199da4bb243585c in term 1.
I20260926 13:50:06.924561 13089 raft_consensus.cc:2454] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543647d1a5eb43f39199da4bb243585c in term 1.
I20260926 13:50:06.924685 13163 leader_election.cc:302] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3625c8f58abd4e1694562858b4a841d8, 543647d1a5eb43f39199da4bb243585c; no voters:
I20260926 13:50:06.924903 13528 raft_consensus.cc:2781] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:06.925161 13528 raft_consensus.cc:686] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 1 LEADER]: Becoming Leader. State: Replica: 543647d1a5eb43f39199da4bb243585c, State: Running, Role: LEADER
I20260926 13:50:06.925433 13528 consensus_queue.cc:237] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } }
I20260926 13:50:06.928009 12933 catalog_manager.cc:5495] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c reported cstate change: term changed from 0 to 1, leader changed from <none> to 543647d1a5eb43f39199da4bb243585c (127.10.236.194). New cstate: current_term: 1 leader_uuid: "543647d1a5eb43f39199da4bb243585c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "543647d1a5eb43f39199da4bb243585c" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 35027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:06.960642 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.956723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51586 (local address 127.10.236.194:35027)
0926 13:50:06.956847 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.956851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.956911 (+ 60us) server_negotiation.cc:407] Connection header received
0926 13:50:06.957066 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.957069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.957121 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.957205 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:06.957476 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.957986 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.958807 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.959121 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.959201 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.959621 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.960217 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.960317 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.960481 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.960535 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:50:06.971894 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.968213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:43393 (local address 127.10.236.194:34493)
0926 13:50:06.968690 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.968705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.968845 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.969103 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.969106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.969113 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:06.969244 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.969250 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.969983 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.969989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.970619 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.970628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.971101 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.971105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.971200 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.971246 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.971593 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.971602 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.971669 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.971754 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":115,"threads_started":1}
I20260926 13:50:06.971879 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.968525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:34493 (local address 127.10.236.195:43393)
0926 13:50:06.968665 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.968670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.968783 (+ 113us) server_negotiation.cc:407] Connection header received
0926 13:50:06.968948 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.968951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.969006 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.969092 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:06.969348 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.969866 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.970745 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.970977 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.971094 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.971389 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.971461 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.971589 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.971677 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.971728 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:06.978528 13353 raft_consensus.cc:1261] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Refusing update from remote peer 543647d1a5eb43f39199da4bb243585c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:06.978525 13089 raft_consensus.cc:1261] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Refusing update from remote peer 543647d1a5eb43f39199da4bb243585c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:06.979040 13528 consensus_queue.cc:1035] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c58b4536044fcf8e5597ca32a26a10" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:06.979249 13552 consensus_queue.cc:1035] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3625c8f58abd4e1694562858b4a841d8" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 43393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:06.997630 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.994150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:34613 (local address 127.10.236.194:35027)
0926 13:50:06.994460 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:06.994462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:06.994472 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:50:06.994505 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:06.994507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:06.994543 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:06.994629 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:06.994858 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.995445 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.996417 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.996705 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.996828 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.997111 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.997193 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.997272 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:06.997397 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 13:50:06.997452 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
I20260926 13:50:06.997629 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:06.994101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:35027 (local address 127.10.236.195:34613)
0926 13:50:06.994242 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:06.994260 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:06.994333 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:06.994639 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:06.994642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:06.994649 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:06.994745 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.994751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.995621 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.995624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.996278 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:06.996288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:06.996832 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:06.996835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:06.996934 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:06.996971 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:06.997301 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:06.997309 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:50:06.997378 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:50:06.997478 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 13:50:06.998359 13494 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 13:50:08.559360 12932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:08.559569 12932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:08.562196 12932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5102595320224e10a7a553c98aa450a6: Sending DeleteTablet for 3 replicas of tablet 635ea9923acd4300a508f4b46857826d
I20260926 13:50:08.562822 13070 tablet_service.cc:1514] Processing DeleteTablet for tablet 635ea9923acd4300a508f4b46857826d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:08 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:54198
I20260926 13:50:08.562808 13208 tablet_service.cc:1514] Processing DeleteTablet for tablet 635ea9923acd4300a508f4b46857826d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:08 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51578
I20260926 13:50:08.562891 13350 tablet_service.cc:1514] Processing DeleteTablet for tablet 635ea9923acd4300a508f4b46857826d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:08 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260926 13:50:08.563191 13583 tablet_replica.cc:330] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10: stopping tablet replica
I20260926 13:50:08.563318 13584 tablet_replica.cc:330] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: stopping tablet replica
I20260926 13:50:08.563346 13583 raft_consensus.cc:2229] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:08.563446 13584 raft_consensus.cc:2229] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:08.563470 13583 raft_consensus.cc:2258] T 635ea9923acd4300a508f4b46857826d P 39c58b4536044fcf8e5597ca32a26a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:08.563628 13584 raft_consensus.cc:2258] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:08.563694 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13005
I20260926 13:50:08.564677 13582 tablet_replica.cc:330] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: stopping tablet replica
I20260926 13:50:08.564857 13582 raft_consensus.cc:2229] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:08.565183 13582 raft_consensus.cc:2258] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:08.565328 13584 ts_tablet_manager.cc:1905] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:08.565883 13582 ts_tablet_manager.cc:1905] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:08.567687 13584 ts_tablet_manager.cc:1918] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:50:08.567785 13584 log.cc:1196] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/635ea9923acd4300a508f4b46857826d
I20260926 13:50:08.568027 13584 ts_tablet_manager.cc:1939] T 635ea9923acd4300a508f4b46857826d P 3625c8f58abd4e1694562858b4a841d8: Deleting consensus metadata
I20260926 13:50:08.569205 13582 ts_tablet_manager.cc:1918] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:50:08.569203 12919 catalog_manager.cc:4841] TS 3625c8f58abd4e1694562858b4a841d8 (127.10.236.195:43393): tablet 635ea9923acd4300a508f4b46857826d (table test-table [id=f86845bff0114203abcc4c08f594e61a]) successfully deleted
I20260926 13:50:08.569291 13582 log.cc:1196] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/635ea9923acd4300a508f4b46857826d
I20260926 13:50:08.569589 13582 ts_tablet_manager.cc:1939] T 635ea9923acd4300a508f4b46857826d P 543647d1a5eb43f39199da4bb243585c: Deleting consensus metadata
I20260926 13:50:08.570441 12919 catalog_manager.cc:4841] TS 543647d1a5eb43f39199da4bb243585c (127.10.236.194:35027): tablet 635ea9923acd4300a508f4b46857826d (table test-table [id=f86845bff0114203abcc4c08f594e61a]) successfully deleted
W20260926 13:50:08.572619 12918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:36421 (error 108)
I20260926 13:50:08.573381 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13145
I20260926 13:50:08.575826 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:08.573375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36421 (local address 127.0.0.1:54220)
0926 13:50:08.575672 (+ 2297us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:08.575742 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2226,"thread_start_us":107,"threads_started":1}
W20260926 13:50:08.576004 12918 catalog_manager.cc:4584] TS 39c58b4536044fcf8e5597ca32a26a10 (127.10.236.193:36421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 635ea9923acd4300a508f4b46857826d: Network error: Client connection negotiation failed: client connection to 127.10.236.193:36421: connect: Connection refused (error 111)
I20260926 13:50:08.580646 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13285
I20260926 13:50:08.590740 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 12901
2026-09-26T13:50:08Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6389 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:08 dist-test-slave-r34j krb5kdc[13589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:08 dist-test-slave-r34j krb5kdc[13589](info): set up 2 sockets
Sep 26 13:50:08 dist-test-slave-r34j krb5kdc[13589](info): commencing operation
krb5kdc: starting...
W20260926 13:50:09.647593 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:09 dist-test-slave-r34j krb5kdc[13589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:09.780452 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34071
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:09.891601 13602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:09.891891 13602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:09.891934 13602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:09.896117 13602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:09.896178 13602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:09.896196 13602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:09.896214 13602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:09.896229 13602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:09.901714 13602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:09.902778 13602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:09.903818 13602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:09.912709 13602 server_base.cc:1034] running on GCE node
W20260926 13:50:09.912643 13610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:09.912637 13608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:09.912621 13607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:09.913594 13602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:09.914759 13602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:09.915984 13602 hybrid_clock.cc:648] HybridClock initialized: now 1790430609915929 us; error 45 us; skew 500 ppm
Sep 26 13:50:09 dist-test-slave-r34j krb5kdc[13589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430609, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:09.919309 13602 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:09.920473 13602 webserver.cc:460] Webserver started at http://127.10.236.254:45407/ using document root <none> and password file <none>
I20260926 13:50:09.921116 13602 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:09.921192 13602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:09.921442 13602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:09.923696 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "d3b2d57adf2b4569952688448570edc1"
format_stamp: "Formatted at 2026-09-26 13:50:09 on dist-test-slave-r34j"
server_key: "16b9d4973c12984f454cce13c6cfe042"
server_key_iv: "8be2e4b72de4f286e4a902c78c885a7f"
server_key_version: "encryptionkey@0"
I20260926 13:50:09.924327 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "d3b2d57adf2b4569952688448570edc1"
format_stamp: "Formatted at 2026-09-26 13:50:09 on dist-test-slave-r34j"
server_key: "16b9d4973c12984f454cce13c6cfe042"
server_key_iv: "8be2e4b72de4f286e4a902c78c885a7f"
server_key_version: "encryptionkey@0"
I20260926 13:50:09.928406 13602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:50:09.930917 13617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:09.932320 13602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:50:09.932490 13602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "d3b2d57adf2b4569952688448570edc1"
format_stamp: "Formatted at 2026-09-26 13:50:09 on dist-test-slave-r34j"
server_key: "16b9d4973c12984f454cce13c6cfe042"
server_key_iv: "8be2e4b72de4f286e4a902c78c885a7f"
server_key_version: "encryptionkey@0"
I20260926 13:50:09.932613 13602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:09.978332 13602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:09.981962 13602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:09.982221 13602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:09.990404 13602 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:34071
I20260926 13:50:09.990437 13669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:34071 every 8 connection(s)
I20260926 13:50:09.991400 13602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:09.994238 13670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:09.997990 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13602
I20260926 13:50:09.998098 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:09.998399 11187 external_mini_cluster.cc:1428] Setting key 3c93febd1638b2656f66e439ece5ca68
I20260926 13:50:10.000360 13670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1: Bootstrap starting.
I20260926 13:50:10.002912 13670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:10.003778 13670 log.cc:824] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:10.005975 13670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1: No bootstrap required, opened a new log
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:10.010104 13670 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2d57adf2b4569952688448570edc1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34071 } }
I20260926 13:50:10.010345 13670 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:10.010409 13670 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b2d57adf2b4569952688448570edc1, State: Initialized, Role: FOLLOWER
I20260926 13:50:10.010948 13670 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2d57adf2b4569952688448570edc1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34071 } }
I20260926 13:50:10.011123 13670 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:10.011206 13670 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:10.011332 13670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:10.012604 13670 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2d57adf2b4569952688448570edc1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34071 } }
I20260926 13:50:10.012996 13670 leader_election.cc:302] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3b2d57adf2b4569952688448570edc1; no voters:
I20260926 13:50:10.013376 13670 leader_election.cc:288] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:10.013468 13675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:10.013715 13675 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [term 1 LEADER]: Becoming Leader. State: Replica: d3b2d57adf2b4569952688448570edc1, State: Running, Role: LEADER
I20260926 13:50:10.014072 13675 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2d57adf2b4569952688448570edc1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34071 } }
I20260926 13:50:10.014773 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.000002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.10.236.254:34071)
0926 13:50:10.000472 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:10.000512 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:10.000562 (+ 50us) server_negotiation.cc:407] Connection header received
0926 13:50:10.001196 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:10.001214 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:10.001561 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:10.001948 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:10.002835 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.004105 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.005239 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.005707 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.005825 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.008788 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:10.008793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:10.008808 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:10.008839 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:10.011442 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.011967 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.011973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.011980 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.012137 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.012437 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.012442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.012443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.012941 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:10.013161 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:10.013465 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 13:50:10.013856 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":169,"threads_started":1}
I20260926 13:50:10.014822 13670 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:10.015844 13677 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3b2d57adf2b4569952688448570edc1. Latest consensus state: current_term: 1 leader_uuid: "d3b2d57adf2b4569952688448570edc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2d57adf2b4569952688448570edc1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34071 } } }
I20260926 13:50:10.015985 13677 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:10.016207 13677 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3b2d57adf2b4569952688448570edc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b2d57adf2b4569952688448570edc1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34071 } } }
I20260926 13:50:10.016291 13677 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:10.017529 13684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:10.020310 13684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:10.025967 13684 catalog_manager.cc:1347] Generated new cluster ID: 25bd2dcb42f24b1d92f165f3798fcd9e
I20260926 13:50:10.026044 13684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:10.049754 13684 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:10.051218 13684 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:10.060197 13684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3b2d57adf2b4569952688448570edc1: Generated new TSK 0
I20260926 13:50:10.060945 13684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:10.091708 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34071
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:10.207017 13696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:10.207283 13696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:10.207326 13696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:10.211248 13696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:10.211306 13696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:10.211362 13696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:10.216789 13696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:10.217814 13696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:10.218772 13696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:10.228416 13702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:10.228556 13696 server_base.cc:1034] running on GCE node
W20260926 13:50:10.228418 13701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:10.228480 13704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:10.229157 13696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:10.229871 13696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:10.231061 13696 hybrid_clock.cc:648] HybridClock initialized: now 1790430610231023 us; error 51 us; skew 500 ppm
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:10.233994 13696 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:10.234920 13696 webserver.cc:460] Webserver started at http://127.10.236.193:45803/ using document root <none> and password file <none>
I20260926 13:50:10.235569 13696 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:10.235664 13696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:10.235893 13696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:10.238014 13696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "a56a50dc012e442ab2ca71f8ffe54252"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "0fce4ba16d6f5f8b5adb82751e12292e"
server_key_iv: "650ef9e7a2cadcd96230afc56df23354"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.238577 13696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "a56a50dc012e442ab2ca71f8ffe54252"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "0fce4ba16d6f5f8b5adb82751e12292e"
server_key_iv: "650ef9e7a2cadcd96230afc56df23354"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.242326 13696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:50:10.244947 13711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.246179 13696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:10.246313 13696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "a56a50dc012e442ab2ca71f8ffe54252"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "0fce4ba16d6f5f8b5adb82751e12292e"
server_key_iv: "650ef9e7a2cadcd96230afc56df23354"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.246423 13696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:10.272703 13696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:10.276319 13696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:10.276527 13696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:10.277164 13696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:10.278007 13696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:10.278053 13696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.278092 13696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:10.278108 13696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.287182 13696 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:34121
I20260926 13:50:10.287207 13824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:34121 every 8 connection(s)
I20260926 13:50:10.288214 13696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:10.288875 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13696
I20260926 13:50:10.288996 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:10.289319 11187 external_mini_cluster.cc:1428] Setting key 25e4618b474575a170f1a85f34380304
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:10.308357 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.290578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:58229 (local address 127.10.236.254:34071)
0926 13:50:10.290786 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:10.290790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:10.291735 (+ 945us) server_negotiation.cc:407] Connection header received
0926 13:50:10.293424 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:10.293427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:10.293473 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:10.293546 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:10.295233 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.295950 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.297927 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.298194 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.298358 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.303792 (+ 5434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:10.303800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:10.303861 (+ 61us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:10.303904 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:10.305629 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.306349 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.306352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.306353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.306440 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.306928 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.306933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.306934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.307221 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:10.307332 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:10.308090 (+ 758us) server_negotiation.cc:299] Negotiation successful
0926 13:50:10.308211 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260926 13:50:10.309737 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.291048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34071 (local address 127.10.236.193:58229)
0926 13:50:10.291570 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:10.291624 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:10.293163 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:10.293784 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:10.293797 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:10.294325 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:10.295063 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:10.295086 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.296094 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.296101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:10.297739 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:10.297751 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.298435 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.298440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:10.298630 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.299400 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:10.299419 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:10.303550 (+ 4131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:10.305759 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:10.305769 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:10.305789 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:10.306230 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:10.306559 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:10.306565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:10.306570 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:10.306777 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:10.307351 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:10.307359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:10.307774 (+ 415us) client_negotiation.cc:769] Sending connection context
0926 13:50:10.308044 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 13:50:10.308470 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260926 13:50:10.311259 13825 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34071
I20260926 13:50:10.311609 13825 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:10.312170 13825 heartbeater.cc:511] Master 127.10.236.254:34071 requested a full tablet report, sending...
I20260926 13:50:10.313956 13634 ts_manager.cc:194] Registered new tserver with Master: a56a50dc012e442ab2ca71f8ffe54252 (127.10.236.193:34121)
I20260926 13:50:10.315317 13634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:58229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:10.322929 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34071
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:10.436558 13830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:10.436815 13830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:10.436896 13830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:10.441002 13830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:10.441063 13830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:10.441149 13830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:10.446451 13830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:10.447525 13830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:10.448562 13830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:10.458815 13835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:10.458882 13838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:10.459018 13830 server_base.cc:1034] running on GCE node
W20260926 13:50:10.458878 13836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:10.459467 13830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:10.460157 13830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:10.461318 13830 hybrid_clock.cc:648] HybridClock initialized: now 1790430610461304 us; error 24 us; skew 500 ppm
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:10.464023 13830 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:10.464892 13830 webserver.cc:460] Webserver started at http://127.10.236.194:38461/ using document root <none> and password file <none>
I20260926 13:50:10.465453 13830 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:10.465523 13830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:10.465731 13830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:10.467823 13830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "a0427043c28a4597ba12ce2d332e3522"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "2b6687172ffb69ae52a7a8b989bcd5e8"
server_key_iv: "6ead2a9611389e7db0482c58801b0460"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.468386 13830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "a0427043c28a4597ba12ce2d332e3522"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "2b6687172ffb69ae52a7a8b989bcd5e8"
server_key_iv: "6ead2a9611389e7db0482c58801b0460"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.472153 13830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:50:10.474567 13845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.475899 13830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:10.476039 13830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "a0427043c28a4597ba12ce2d332e3522"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "2b6687172ffb69ae52a7a8b989bcd5e8"
server_key_iv: "6ead2a9611389e7db0482c58801b0460"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.476154 13830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:10.507177 13830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:10.510921 13830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:10.511121 13830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:10.511819 13830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:10.512735 13830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:10.512787 13830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.512831 13830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:10.512849 13830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.522863 13830 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:40411
I20260926 13:50:10.522881 13958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:40411 every 8 connection(s)
I20260926 13:50:10.523945 13830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:10.529994 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13830
I20260926 13:50:10.530114 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:10.530422 11187 external_mini_cluster.cc:1428] Setting key 014cad3d05d14384788d8293a396ffc2
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:10.538139 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.525789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:55737 (local address 127.10.236.254:34071)
0926 13:50:10.525942 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:10.525947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:10.526858 (+ 911us) server_negotiation.cc:407] Connection header received
0926 13:50:10.528195 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:10.528198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:10.528247 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:10.528324 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:10.529503 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.530247 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.531575 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.531808 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.531940 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.534359 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:10.534363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:10.534365 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:10.534387 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:10.535973 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.536556 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.536560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.536562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.536647 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.537025 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.537028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.537029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.537215 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:10.537331 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:10.537855 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 13:50:10.537975 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 13:50:10.538991 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.526222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34071 (local address 127.10.236.194:55737)
0926 13:50:10.526684 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:10.526739 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:10.527962 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:10.528494 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:10.528500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:10.528866 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:10.529346 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:10.529359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.530373 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.530376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:10.531415 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:10.531424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.531968 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.531971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:10.532115 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.532735 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:10.532751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:10.534186 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:10.536086 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:10.536092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:10.536108 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:10.536448 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:10.536745 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:10.536748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:10.536750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:10.536936 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:10.537319 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:10.537326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:10.537611 (+ 285us) client_negotiation.cc:769] Sending connection context
0926 13:50:10.537804 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 13:50:10.538147 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:10.540422 13959 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34071
I20260926 13:50:10.540683 13959 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:10.541244 13959 heartbeater.cc:511] Master 127.10.236.254:34071 requested a full tablet report, sending...
I20260926 13:50:10.542294 13634 ts_manager.cc:194] Registered new tserver with Master: a0427043c28a4597ba12ce2d332e3522 (127.10.236.194:40411)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:50:10.543059 13634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:55737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:10.559607 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34071
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:10.676363 13964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:10.676656 13964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:10.676702 13964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:10.680935 13964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:10.681013 13964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:10.681075 13964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:10.686674 13964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:10.687925 13964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:10.689007 13964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:10.700034 13969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:10.700234 13972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:10.700305 13964 server_base.cc:1034] running on GCE node
W20260926 13:50:10.700443 13970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:10.700904 13964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:10.701630 13964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:10.702805 13964 hybrid_clock.cc:648] HybridClock initialized: now 1790430610702771 us; error 47 us; skew 500 ppm
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:10.706066 13964 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:10.707123 13964 webserver.cc:460] Webserver started at http://127.10.236.195:45427/ using document root <none> and password file <none>
I20260926 13:50:10.707785 13964 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:10.707851 13964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:10.708077 13964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:10.710212 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "55eedc5759f34a1e9c2eb39c578222d4"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "2fc3ceaf0c07ccd27c7aa9f92e747380"
server_key_iv: "f93aaf77dc5b622165babd4bb8cbf53d"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.710800 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "55eedc5759f34a1e9c2eb39c578222d4"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "2fc3ceaf0c07ccd27c7aa9f92e747380"
server_key_iv: "f93aaf77dc5b622165babd4bb8cbf53d"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.714726 13964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:10.717691 13979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.719138 13964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 13:50:10.719247 13964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "55eedc5759f34a1e9c2eb39c578222d4"
format_stamp: "Formatted at 2026-09-26 13:50:10 on dist-test-slave-r34j"
server_key: "2fc3ceaf0c07ccd27c7aa9f92e747380"
server_key_iv: "f93aaf77dc5b622165babd4bb8cbf53d"
server_key_version: "encryptionkey@0"
I20260926 13:50:10.719420 13964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:10.741858 13964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:10.745494 13964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:10.745723 13964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:10.746405 13964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:10.747273 13964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:10.747323 13964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.747363 13964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:10.747380 13964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:10.756989 13964 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:39233
I20260926 13:50:10.757055 14092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:39233 every 8 connection(s)
I20260926 13:50:10.757944 13964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:10.766754 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 13964
I20260926 13:50:10.766896 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:10.767244 11187 external_mini_cluster.cc:1428] Setting key 05e9e485262de6f8565083d3045e59aa
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[13589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430610, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:10.774417 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.760261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:35837 (local address 127.10.236.254:34071)
0926 13:50:10.760458 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:10.760464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:10.761580 (+ 1116us) server_negotiation.cc:407] Connection header received
0926 13:50:10.762948 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:10.762953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:10.763033 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:10.763256 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:10.764696 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.765284 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.766561 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.766918 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.767040 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.770633 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:10.770636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:10.770638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:10.770665 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:10.772218 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.772791 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.772795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.772797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.772880 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:10.773240 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:10.773244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:10.773246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:10.773455 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:10.773561 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:10.774112 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 13:50:10.774242 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 13:50:10.775290 14095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:10.760563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34071 (local address 127.10.236.195:35837)
0926 13:50:10.761383 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:10.761437 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:10.762644 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:10.763451 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:10.763459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:10.763976 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:10.764517 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:10.764533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.765458 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.765461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:10.766416 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:10.766425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:10.767123 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:10.767126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:10.767342 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:10.768261 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:10.768297 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:10.770447 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:10.772343 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:10.772350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:10.772369 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:10.772663 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:10.772972 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:10.772975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:10.772978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:10.773139 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:10.773573 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:10.773582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:10.773869 (+ 287us) client_negotiation.cc:769] Sending connection context
0926 13:50:10.774075 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 13:50:10.774429 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":254,"threads_started":1}
I20260926 13:50:10.776638 14093 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34071
I20260926 13:50:10.776928 14093 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:10.777457 14093 heartbeater.cc:511] Master 127.10.236.254:34071 requested a full tablet report, sending...
I20260926 13:50:10.778733 13634 ts_manager.cc:194] Registered new tserver with Master: 55eedc5759f34a1e9c2eb39c578222d4 (127.10.236.195:39233)
I20260926 13:50:10.779423 13634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:35837
I20260926 13:50:10.782568 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 13:50:10.796801 13673 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:10.794924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.10.236.254:34071)
0926 13:50:10.795204 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:10.795209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:10.795392 (+ 183us) server_negotiation.cc:407] Connection header received
0926 13:50:10.795660 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:10.795664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:10.795712 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:10.795841 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:10.796680 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55326: BlockingRecv error: recv got EOF from 127.0.0.1:55326 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
I20260926 13:50:10.798187 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13696
I20260926 13:50:10.804351 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13830
I20260926 13:50:10.810617 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13964
I20260926 13:50:10.816264 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 13602
2026-09-26T13:50:10Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2222 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[14105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[14105](info): set up 2 sockets
Sep 26 13:50:10 dist-test-slave-r34j krb5kdc[14105](info): commencing operation
krb5kdc: starting...
W20260926 13:50:11.863617 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:11 dist-test-slave-r34j krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:11.992077 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44649
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:12.107111 14118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:12.107362 14118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:12.107399 14118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:12.111368 14118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:12.111425 14118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:12.111444 14118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:12.111461 14118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:12.111529 14118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:12.116688 14118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:12.117713 14118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:12.118746 14118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:12.127171 14124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:12.127257 14123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:12.127179 14126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:12.127373 14118 server_base.cc:1034] running on GCE node
I20260926 13:50:12.128017 14118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:12.129091 14118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:12.130252 14118 hybrid_clock.cc:648] HybridClock initialized: now 1790430612130225 us; error 40 us; skew 500 ppm
Sep 26 13:50:12 dist-test-slave-r34j krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430612, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:12.139608 14118 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:12.140750 14118 webserver.cc:460] Webserver started at http://127.10.236.254:44569/ using document root <none> and password file <none>
I20260926 13:50:12.141369 14118 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:12.141469 14118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:12.141722 14118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:12.144011 14118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "03271084d4a74e6fb9e692613bd489a4"
format_stamp: "Formatted at 2026-09-26 13:50:12 on dist-test-slave-r34j"
server_key: "86e5951a11583fc483dde8df6fa160b6"
server_key_iv: "579ab3f5a4a99eb3f6aa5c5c7ed44109"
server_key_version: "encryptionkey@0"
I20260926 13:50:12.144632 14118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "03271084d4a74e6fb9e692613bd489a4"
format_stamp: "Formatted at 2026-09-26 13:50:12 on dist-test-slave-r34j"
server_key: "86e5951a11583fc483dde8df6fa160b6"
server_key_iv: "579ab3f5a4a99eb3f6aa5c5c7ed44109"
server_key_version: "encryptionkey@0"
I20260926 13:50:12.148689 14118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:12.151338 14133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:12.152655 14118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:12.152817 14118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "03271084d4a74e6fb9e692613bd489a4"
format_stamp: "Formatted at 2026-09-26 13:50:12 on dist-test-slave-r34j"
server_key: "86e5951a11583fc483dde8df6fa160b6"
server_key_iv: "579ab3f5a4a99eb3f6aa5c5c7ed44109"
server_key_version: "encryptionkey@0"
I20260926 13:50:12.152945 14118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:12.185407 14118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:12.188486 14118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:12.188717 14118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:12.195382 14118 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44649
I20260926 13:50:12.195739 14185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44649 every 8 connection(s)
I20260926 13:50:12.196460 14118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:12.199236 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14118
I20260926 13:50:12.199361 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:12.199368 14186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:12.199802 11187 external_mini_cluster.cc:1428] Setting key accfbf303b7215eea9f7c2f5458b4a9c
I20260926 13:50:12.206264 14186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4: Bootstrap starting.
I20260926 13:50:12.209107 14186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4: Neither blocks nor log segments found. Creating new log.
Sep 26 13:50:12 dist-test-slave-r34j krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:12.209851 14186 log.cc:824] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:12.212108 14186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4: No bootstrap required, opened a new log
I20260926 13:50:12.214437 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:12.201180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.10.236.254:44649)
0926 13:50:12.201621 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:12.201637 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:12.201671 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:50:12.202588 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:12.202607 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:12.202939 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:12.203232 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:12.204204 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:12.205199 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:12.206322 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:12.206832 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:12.206947 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:12.209720 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:12.209725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:12.209738 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:12.209765 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:12.211551 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:12.212080 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:12.212087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:12.212092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:12.212260 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:12.212555 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:12.212559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:12.212563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:12.212958 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:12.213167 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:12.213437 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 13:50:12.213725 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":149,"threads_started":1}
I20260926 13:50:12.216135 14186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03271084d4a74e6fb9e692613bd489a4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44649 } }
I20260926 13:50:12.216420 14186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:12.216478 14186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03271084d4a74e6fb9e692613bd489a4, State: Initialized, Role: FOLLOWER
I20260926 13:50:12.216872 14186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03271084d4a74e6fb9e692613bd489a4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44649 } }
I20260926 13:50:12.217042 14186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:12.217114 14186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:12.217198 14186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:12.217628 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:50:12.218173 14186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03271084d4a74e6fb9e692613bd489a4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44649 } }
I20260926 13:50:12.218518 14186 leader_election.cc:302] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03271084d4a74e6fb9e692613bd489a4; no voters:
I20260926 13:50:12.218855 14186 leader_election.cc:288] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:12.218941 14191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:12.219123 14191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [term 1 LEADER]: Becoming Leader. State: Replica: 03271084d4a74e6fb9e692613bd489a4, State: Running, Role: LEADER
I20260926 13:50:12.219410 14191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03271084d4a74e6fb9e692613bd489a4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44649 } }
I20260926 13:50:12.220160 14186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:12.220965 14192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03271084d4a74e6fb9e692613bd489a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03271084d4a74e6fb9e692613bd489a4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44649 } } }
I20260926 13:50:12.221117 14192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:12.221356 14192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03271084d4a74e6fb9e692613bd489a4. Latest consensus state: current_term: 1 leader_uuid: "03271084d4a74e6fb9e692613bd489a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03271084d4a74e6fb9e692613bd489a4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44649 } } }
I20260926 13:50:12.221450 14192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:12.222623 14200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:12.225950 14200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:12.232939 14200 catalog_manager.cc:1347] Generated new cluster ID: feb532245b6142dda69a66a182c64519
I20260926 13:50:12.233019 14200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:12.237780 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:12.231661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.10.236.254:44649)
0926 13:50:12.231801 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:12.231807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:12.232053 (+ 246us) server_negotiation.cc:407] Connection header received
0926 13:50:12.232229 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:12.232233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:12.232289 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:12.232394 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:12.233248 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:12.234076 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:12.235276 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:12.235683 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:12.235798 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:12.236745 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:12.236750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:12.236753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:12.237319 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:12.237354 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:12.237374 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:12.237463 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:12.237575 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 13:50:12.237634 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:50:12.252022 14200 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:12.253242 14200 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:12.259785 14200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03271084d4a74e6fb9e692613bd489a4: Generated new TSK 0
I20260926 13:50:12.260540 14200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:12.274137 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14118
2026-09-26T13:50:12Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1455 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:12 dist-test-slave-r34j krb5kdc[14219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:12 dist-test-slave-r34j krb5kdc[14219](info): set up 2 sockets
Sep 26 13:50:12 dist-test-slave-r34j krb5kdc[14219](info): commencing operation
krb5kdc: starting...
W20260926 13:50:13.319532 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:13 dist-test-slave-r34j krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:13.452088 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33957
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:33957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:13.564517 14232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:13.564808 14232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:13.564850 14232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:13.568861 14232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:13.568921 14232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:13.568974 14232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:13.568996 14232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:13.569011 14232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:13.574316 14232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:33957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:13.575371 14232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:13.576447 14232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:13.585268 14240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:13.585258 14237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:13.585263 14238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:13.585314 14232 server_base.cc:1034] running on GCE node
I20260926 13:50:13.586244 14232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:13.587407 14232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:13.588596 14232 hybrid_clock.cc:648] HybridClock initialized: now 1790430613588572 us; error 37 us; skew 500 ppm
Sep 26 13:50:13 dist-test-slave-r34j krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:13.591464 14232 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:13.592427 14232 webserver.cc:460] Webserver started at http://127.10.236.254:33375/ using document root <none> and password file <none>
I20260926 13:50:13.593043 14232 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:13.593106 14232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:13.593318 14232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:13.595424 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "19d955d42a2b45e3acfe61bad168700e"
format_stamp: "Formatted at 2026-09-26 13:50:13 on dist-test-slave-r34j"
server_key: "08fc70df566e2a2950a3c10f016e9282"
server_key_iv: "f754e709f2c59c42e4843fa3791e57f2"
server_key_version: "encryptionkey@0"
I20260926 13:50:13.596069 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "19d955d42a2b45e3acfe61bad168700e"
format_stamp: "Formatted at 2026-09-26 13:50:13 on dist-test-slave-r34j"
server_key: "08fc70df566e2a2950a3c10f016e9282"
server_key_iv: "f754e709f2c59c42e4843fa3791e57f2"
server_key_version: "encryptionkey@0"
I20260926 13:50:13.599957 14232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:50:13.602384 14247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:13.604040 14232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:13.604163 14232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "19d955d42a2b45e3acfe61bad168700e"
format_stamp: "Formatted at 2026-09-26 13:50:13 on dist-test-slave-r34j"
server_key: "08fc70df566e2a2950a3c10f016e9282"
server_key_iv: "f754e709f2c59c42e4843fa3791e57f2"
server_key_version: "encryptionkey@0"
I20260926 13:50:13.604301 14232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:13.637296 14232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:13.640435 14232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:13.640645 14232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:13.647840 14232 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:33957
I20260926 13:50:13.648113 14299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:33957 every 8 connection(s)
I20260926 13:50:13.648975 14232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:13.651818 14300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:13.657346 14300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e: Bootstrap starting.
I20260926 13:50:13.658957 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14232
I20260926 13:50:13.659065 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:13.659351 11187 external_mini_cluster.cc:1428] Setting key 22d65af57c4400037a89eb252b44b8a8
I20260926 13:50:13.660142 14300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:13.660977 14300 log.cc:824] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:13.663204 14300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e: No bootstrap required, opened a new log
I20260926 13:50:13.666621 14300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d955d42a2b45e3acfe61bad168700e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33957 } }
I20260926 13:50:13.666860 14300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:13.666926 14300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19d955d42a2b45e3acfe61bad168700e, State: Initialized, Role: FOLLOWER
I20260926 13:50:13.667390 14300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d955d42a2b45e3acfe61bad168700e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33957 } }
I20260926 13:50:13.667634 14300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:13.667758 14300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:13.667874 14300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 0 FOLLOWER]: Advancing to term 1
Sep 26 13:50:13 dist-test-slave-r34j krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:13.669106 14300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d955d42a2b45e3acfe61bad168700e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33957 } }
I20260926 13:50:13.669479 14300 leader_election.cc:302] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19d955d42a2b45e3acfe61bad168700e; no voters:
I20260926 13:50:13.669843 14300 leader_election.cc:288] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:13.669920 14305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:13.670127 14305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [term 1 LEADER]: Becoming Leader. State: Replica: 19d955d42a2b45e3acfe61bad168700e, State: Running, Role: LEADER
I20260926 13:50:13.670449 14305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d955d42a2b45e3acfe61bad168700e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33957 } }
I20260926 13:50:13.671253 14300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:13.671769 14307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19d955d42a2b45e3acfe61bad168700e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d955d42a2b45e3acfe61bad168700e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33957 } } }
I20260926 13:50:13.671900 14307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:13.672672 14306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19d955d42a2b45e3acfe61bad168700e. Latest consensus state: current_term: 1 leader_uuid: "19d955d42a2b45e3acfe61bad168700e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d955d42a2b45e3acfe61bad168700e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33957 } } }
I20260926 13:50:13.672791 14306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:13.673115 14309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:13.675915 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:13.660529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.10.236.254:33957)
0926 13:50:13.661050 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:13.661071 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:13.661113 (+ 42us) server_negotiation.cc:407] Connection header received
0926 13:50:13.661906 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:13.661930 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:13.662319 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:13.662680 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:13.663409 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:13.664518 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:13.665399 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:13.665867 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:13.666059 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:13.669190 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:13.669199 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:13.669218 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:13.669257 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:13.671776 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:13.672379 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:13.672386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:13.672392 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:13.672563 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:13.672939 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:13.672943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:13.672945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:13.673505 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:13.674371 (+ 866us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:13.674538 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 13:50:13.674877 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":149,"threads_started":1}
I20260926 13:50:13.676883 14309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:13.682730 14309 catalog_manager.cc:1347] Generated new cluster ID: 2f36ef43d1ef49bf8eaafc933c3e15ec
I20260926 13:50:13.682806 14309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:13.723781 14309 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:13.724893 14309 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:13.735443 14309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19d955d42a2b45e3acfe61bad168700e: Generated new TSK 0
I20260926 13:50:13.736444 14309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:13.775876 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33957
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:13.888535 14326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:13.888854 14326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:13.888932 14326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:13.893110 14326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:13.893198 14326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:13.893316 14326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:13.898703 14326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:13.899912 14326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:13.900949 14326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:13.911289 14334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:13.911370 14326 server_base.cc:1034] running on GCE node
W20260926 13:50:13.911320 14332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:13.911535 14331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:13.912003 14326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:13.912653 14326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:13.913846 14326 hybrid_clock.cc:648] HybridClock initialized: now 1790430613913817 us; error 40 us; skew 500 ppm
Sep 26 13:50:13 dist-test-slave-r34j krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:13.916725 14326 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:13.917615 14326 webserver.cc:460] Webserver started at http://127.10.236.193:32877/ using document root <none> and password file <none>
I20260926 13:50:13.918196 14326 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:13.918289 14326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:13.918519 14326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:13.920677 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "625d7103844a4b20b105ad3f27e0bdf7"
format_stamp: "Formatted at 2026-09-26 13:50:13 on dist-test-slave-r34j"
server_key: "3bca7b4f33736ed1dc803d3c2e24c8ab"
server_key_iv: "33141629c97770a1ec658f0456972dab"
server_key_version: "encryptionkey@0"
I20260926 13:50:13.921325 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "625d7103844a4b20b105ad3f27e0bdf7"
format_stamp: "Formatted at 2026-09-26 13:50:13 on dist-test-slave-r34j"
server_key: "3bca7b4f33736ed1dc803d3c2e24c8ab"
server_key_iv: "33141629c97770a1ec658f0456972dab"
server_key_version: "encryptionkey@0"
I20260926 13:50:13.925179 14326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:13.927629 14341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:13.928897 14326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:13.928994 14326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "625d7103844a4b20b105ad3f27e0bdf7"
format_stamp: "Formatted at 2026-09-26 13:50:13 on dist-test-slave-r34j"
server_key: "3bca7b4f33736ed1dc803d3c2e24c8ab"
server_key_iv: "33141629c97770a1ec658f0456972dab"
server_key_version: "encryptionkey@0"
I20260926 13:50:13.929082 14326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:13.942524 14326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:13.945459 14326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:13.945608 14326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:13.946130 14326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:13.946870 14326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:13.946918 14326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:13.946955 14326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:13.946977 14326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:13.955614 14326 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:46811
I20260926 13:50:13.955693 14454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:46811 every 8 connection(s)
I20260926 13:50:13.956578 14326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:13.962657 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14326
I20260926 13:50:13.962793 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:13.963096 11187 external_mini_cluster.cc:1428] Setting key 11e05165195944fbf6aa1716040ee281
Sep 26 13:50:13 dist-test-slave-r34j krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:13.971365 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:13.958478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:39761 (local address 127.10.236.254:33957)
0926 13:50:13.958627 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:13.958630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:13.959460 (+ 830us) server_negotiation.cc:407] Connection header received
0926 13:50:13.960854 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:13.960858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:13.960921 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:13.961057 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:13.962330 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:13.963136 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:13.964544 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:13.964798 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:13.964929 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:13.967736 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:13.967739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:13.967741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:13.967764 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:13.969274 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:13.969816 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:13.969818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:13.969820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:13.969896 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:13.970267 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:13.970270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:13.970272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:13.970454 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:13.970562 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:13.971065 (+ 503us) server_negotiation.cc:299] Negotiation successful
0926 13:50:13.971193 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:50:13.972273 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:13.958796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33957 (local address 127.10.236.193:39761)
0926 13:50:13.959282 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:13.959331 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:13.960632 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:13.961231 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:13.961238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:13.961673 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:13.962167 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:13.962180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:13.963262 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:13.963265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:13.964418 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:13.964427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:13.964968 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:13.964972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:13.965151 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:13.965786 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:13.965803 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:13.967560 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:13.969394 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:13.969401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:13.969425 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:13.969681 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:13.969999 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:13.970007 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:13.970009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:13.970163 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:13.970559 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:13.970566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:13.970820 (+ 254us) client_negotiation.cc:769] Sending connection context
0926 13:50:13.971010 (+ 190us) client_negotiation.cc:240] Negotiation successful
0926 13:50:13.971398 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:50:13.973704 14455 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33957
I20260926 13:50:13.973978 14455 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:13.974514 14455 heartbeater.cc:511] Master 127.10.236.254:33957 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:50:13.976369 14264 ts_manager.cc:194] Registered new tserver with Master: 625d7103844a4b20b105ad3f27e0bdf7 (127.10.236.193:46811)
I20260926 13:50:13.977977 14264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:39761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:13.995967 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33957
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:14.110338 14460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:14.110757 14460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:14.110846 14460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:14.117863 14460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:14.117956 14460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:14.118057 14460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:14.123926 14460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:14.125087 14460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:14.126157 14460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:14.136261 14466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:14.136288 14465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:14.136375 14460 server_base.cc:1034] running on GCE node
W20260926 13:50:14.136237 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:14.136910 14460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:14.137621 14460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:14.138798 14460 hybrid_clock.cc:648] HybridClock initialized: now 1790430614138778 us; error 29 us; skew 500 ppm
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:14.141757 14460 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:14.142688 14460 webserver.cc:460] Webserver started at http://127.10.236.194:43217/ using document root <none> and password file <none>
I20260926 13:50:14.143281 14460 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:14.143375 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:14.143682 14460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:14.145795 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "a0ac669d1afe40828a360ea974795926"
format_stamp: "Formatted at 2026-09-26 13:50:14 on dist-test-slave-r34j"
server_key: "a59fa4815619ad903438ac3787f15fbc"
server_key_iv: "fbaab212a40b5b23d8ebdbd250aaaef2"
server_key_version: "encryptionkey@0"
I20260926 13:50:14.146407 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "a0ac669d1afe40828a360ea974795926"
format_stamp: "Formatted at 2026-09-26 13:50:14 on dist-test-slave-r34j"
server_key: "a59fa4815619ad903438ac3787f15fbc"
server_key_iv: "fbaab212a40b5b23d8ebdbd250aaaef2"
server_key_version: "encryptionkey@0"
I20260926 13:50:14.150579 14460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:14.153216 14475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:14.154417 14460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:14.154556 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "a0ac669d1afe40828a360ea974795926"
format_stamp: "Formatted at 2026-09-26 13:50:14 on dist-test-slave-r34j"
server_key: "a59fa4815619ad903438ac3787f15fbc"
server_key_iv: "fbaab212a40b5b23d8ebdbd250aaaef2"
server_key_version: "encryptionkey@0"
I20260926 13:50:14.154680 14460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:14.186770 14460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:14.190225 14460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:14.190424 14460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:14.191164 14460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:14.192114 14460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:14.192162 14460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:14.192226 14460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:14.192265 14460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:14.201333 14460 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:36697
I20260926 13:50:14.201355 14588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:36697 every 8 connection(s)
I20260926 13:50:14.202333 14460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:14.202693 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14460
I20260926 13:50:14.202821 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:14.203133 11187 external_mini_cluster.cc:1428] Setting key 8fb58eab7c3387ba1e12861daddb7596
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:14.221369 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:14.204503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:47497 (local address 127.10.236.254:33957)
0926 13:50:14.204625 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:14.204630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:14.205739 (+ 1109us) server_negotiation.cc:407] Connection header received
0926 13:50:14.207749 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:14.207753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:14.207806 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:14.207933 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:14.209330 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.209880 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.211145 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.211385 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.211520 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:14.217129 (+ 5609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:14.217132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:14.217134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:14.217161 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:14.218943 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:14.219696 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:14.219699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:14.219700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:14.219779 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:14.220209 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:14.220211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:14.220212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:14.220389 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:14.220497 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:14.221043 (+ 546us) server_negotiation.cc:299] Negotiation successful
0926 13:50:14.221190 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:50:14.222224 14591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:14.205045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33957 (local address 127.10.236.194:47497)
0926 13:50:14.205572 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:14.205633 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:14.207361 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:14.208141 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:14.208149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:14.208578 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:14.209180 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:14.209194 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.210017 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.210023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:14.211012 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:14.211021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.211574 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.211578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:14.211745 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:14.212398 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:14.212415 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:14.216916 (+ 4501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:14.219117 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:14.219126 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:14.219148 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:14.219591 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:14.219877 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:14.219883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:14.219887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:14.220096 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:14.220500 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:14.220510 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:14.220787 (+ 277us) client_negotiation.cc:769] Sending connection context
0926 13:50:14.220987 (+ 200us) client_negotiation.cc:240] Negotiation successful
0926 13:50:14.221333 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":147,"threads_started":1}
I20260926 13:50:14.223450 14589 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33957
I20260926 13:50:14.223754 14589 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:14.224267 14589 heartbeater.cc:511] Master 127.10.236.254:33957 requested a full tablet report, sending...
I20260926 13:50:14.225374 14264 ts_manager.cc:194] Registered new tserver with Master: a0ac669d1afe40828a360ea974795926 (127.10.236.194:36697)
I20260926 13:50:14.225960 14264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:47497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:14.235903 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33957
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:14.348986 14594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:14.349246 14594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:14.349288 14594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:14.353417 14594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:14.353482 14594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:14.353542 14594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:14.358906 14594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:14.360020 14594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:14.361039 14594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:14.370639 14600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:14.370739 14602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:14.371090 14599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:14.371147 14594 server_base.cc:1034] running on GCE node
I20260926 13:50:14.371644 14594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:14.372596 14594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:14.373793 14594 hybrid_clock.cc:648] HybridClock initialized: now 1790430614373770 us; error 35 us; skew 500 ppm
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:14.376777 14594 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:14.377638 14594 webserver.cc:460] Webserver started at http://127.10.236.195:37451/ using document root <none> and password file <none>
I20260926 13:50:14.378187 14594 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:14.378247 14594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:14.378414 14594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:14.380613 14594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "8ce9d361c941455a93af0673273ea374"
format_stamp: "Formatted at 2026-09-26 13:50:14 on dist-test-slave-r34j"
server_key: "62f22c7fe25e69aaee388af7af8a0955"
server_key_iv: "593c8501c4fc0cf650cc5dd07c62e6a0"
server_key_version: "encryptionkey@0"
I20260926 13:50:14.381145 14594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "8ce9d361c941455a93af0673273ea374"
format_stamp: "Formatted at 2026-09-26 13:50:14 on dist-test-slave-r34j"
server_key: "62f22c7fe25e69aaee388af7af8a0955"
server_key_iv: "593c8501c4fc0cf650cc5dd07c62e6a0"
server_key_version: "encryptionkey@0"
I20260926 13:50:14.385066 14594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:14.387570 14609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:14.388935 14594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:14.389086 14594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "8ce9d361c941455a93af0673273ea374"
format_stamp: "Formatted at 2026-09-26 13:50:14 on dist-test-slave-r34j"
server_key: "62f22c7fe25e69aaee388af7af8a0955"
server_key_iv: "593c8501c4fc0cf650cc5dd07c62e6a0"
server_key_version: "encryptionkey@0"
I20260926 13:50:14.389200 14594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:14.409163 14594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:14.412808 14594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:14.413066 14594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:14.413750 14594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:14.414644 14594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:14.414718 14594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:14.414786 14594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:14.414831 14594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:14.424306 14594 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:42579
I20260926 13:50:14.424331 14722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:42579 every 8 connection(s)
I20260926 13:50:14.425258 14594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:14.433153 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14594
I20260926 13:50:14.433310 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:14.433597 11187 external_mini_cluster.cc:1428] Setting key 48d80655c8744380c412a0dd85a0237f
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:14.439625 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:14.427195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45171 (local address 127.10.236.254:33957)
0926 13:50:14.427342 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:14.427345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:14.428235 (+ 890us) server_negotiation.cc:407] Connection header received
0926 13:50:14.429629 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:14.429633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:14.429694 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:14.429798 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:14.431010 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.431602 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.432780 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.433124 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.433264 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:14.436039 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:14.436043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:14.436046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:14.436074 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:14.437548 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:14.438071 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:14.438075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:14.438077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:14.438161 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:14.438507 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:14.438512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:14.438514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:14.438702 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:14.438813 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:14.439315 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 13:50:14.439435 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:14.440454 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:14.427603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33957 (local address 127.10.236.195:45171)
0926 13:50:14.428059 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:14.428102 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:14.429390 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:14.429981 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:14.429988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:14.430364 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:14.430854 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:14.430866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.431749 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.431752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:14.432651 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:14.432660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.433393 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.433399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:14.433585 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:14.434310 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:14.434334 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:14.435835 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:14.437679 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:14.437685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:14.437701 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:14.437967 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:14.438265 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:14.438268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:14.438270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:14.438411 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:14.438809 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:14.438815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:14.439070 (+ 255us) client_negotiation.cc:769] Sending connection context
0926 13:50:14.439275 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 13:50:14.439632 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260926 13:50:14.441609 14723 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33957
I20260926 13:50:14.441825 14723 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:14.442312 14723 heartbeater.cc:511] Master 127.10.236.254:33957 requested a full tablet report, sending...
I20260926 13:50:14.443315 14264 ts_manager.cc:194] Registered new tserver with Master: 8ce9d361c941455a93af0673273ea374 (127.10.236.195:42579)
I20260926 13:50:14.443913 14264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:45171
I20260926 13:50:14.447702 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:14.474289 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:14.465238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.10.236.254:33957)
0926 13:50:14.465447 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:14.465451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:14.465515 (+ 64us) server_negotiation.cc:407] Connection header received
0926 13:50:14.465637 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:14.465640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:14.465686 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:14.465764 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:14.466521 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.467077 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.468053 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:14.468267 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:14.468369 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:14.470896 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:14.470900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:14.470902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:14.470928 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:14.472480 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:14.472968 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:14.472971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:14.472972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:14.473059 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:14.473356 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:14.473359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:14.473360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:14.473611 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:14.473756 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:14.473953 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 13:50:14.474091 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260926 13:50:14.474694 14264 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56020
I20260926 13:50:14.475641 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14326
I20260926 13:50:14.481520 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14460
I20260926 13:50:14.486655 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14594
I20260926 13:50:14.491608 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14232
2026-09-26T13:50:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2219 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14735](info): set up 2 sockets
Sep 26 13:50:14 dist-test-slave-r34j krb5kdc[14735](info): commencing operation
krb5kdc: starting...
W20260926 13:50:15.563513 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:15 dist-test-slave-r34j krb5kdc[14735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:15.690800 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45853
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:45853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:15.801599 14748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:15.801939 14748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:15.802026 14748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:15.806162 14748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:15.806278 14748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:15.806316 14748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:15.806356 14748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:15.806386 14748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:15.812001 14748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:45853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:15.813227 14748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:15.814360 14748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:15.823335 14754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:15.823386 14748 server_base.cc:1034] running on GCE node
W20260926 13:50:15.823333 14756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:15.823333 14753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:15.824254 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:15.825440 14748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:15.826619 14748 hybrid_clock.cc:648] HybridClock initialized: now 1790430615826597 us; error 36 us; skew 500 ppm
Sep 26 13:50:15 dist-test-slave-r34j krb5kdc[14735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430615, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:15.829530 14748 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:15.830451 14748 webserver.cc:460] Webserver started at http://127.10.236.254:42367/ using document root <none> and password file <none>
I20260926 13:50:15.831034 14748 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:15.831127 14748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:15.831359 14748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:15.833488 14748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "f34490aa2254476ab0e07239dcfbc736"
format_stamp: "Formatted at 2026-09-26 13:50:15 on dist-test-slave-r34j"
server_key: "24d9479c2dbf928e5d62645edbf6bb46"
server_key_iv: "84ee3852a9f2cc5784a6c641e55c4d17"
server_key_version: "encryptionkey@0"
I20260926 13:50:15.834048 14748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "f34490aa2254476ab0e07239dcfbc736"
format_stamp: "Formatted at 2026-09-26 13:50:15 on dist-test-slave-r34j"
server_key: "24d9479c2dbf928e5d62645edbf6bb46"
server_key_iv: "84ee3852a9f2cc5784a6c641e55c4d17"
server_key_version: "encryptionkey@0"
I20260926 13:50:15.838070 14748 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:15.840735 14763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:15.842044 14748 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:15.842231 14748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "f34490aa2254476ab0e07239dcfbc736"
format_stamp: "Formatted at 2026-09-26 13:50:15 on dist-test-slave-r34j"
server_key: "24d9479c2dbf928e5d62645edbf6bb46"
server_key_iv: "84ee3852a9f2cc5784a6c641e55c4d17"
server_key_version: "encryptionkey@0"
I20260926 13:50:15.842352 14748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:15.856443 14748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:15.859910 14748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:15.860177 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:15.867509 14748 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:45853
I20260926 13:50:15.867658 14815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:45853 every 8 connection(s)
I20260926 13:50:15.868551 14748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:15.871445 14816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:15.877182 14816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736: Bootstrap starting.
I20260926 13:50:15.877620 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14748
I20260926 13:50:15.877770 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:15.878123 11187 external_mini_cluster.cc:1428] Setting key 0ef36db60795b8a477484e74f1dc916c
I20260926 13:50:15.880050 14816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:15.880795 14816 log.cc:824] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:15.883077 14816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736: No bootstrap required, opened a new log
I20260926 13:50:15.887104 14816 raft_consensus.cc:348] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34490aa2254476ab0e07239dcfbc736" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45853 } }
I20260926 13:50:15.887364 14816 raft_consensus.cc:374] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 26 13:50:15 dist-test-slave-r34j krb5kdc[14735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:15.887450 14816 raft_consensus.cc:729] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34490aa2254476ab0e07239dcfbc736, State: Initialized, Role: FOLLOWER
I20260926 13:50:15.888022 14816 consensus_queue.cc:260] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34490aa2254476ab0e07239dcfbc736" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45853 } }
I20260926 13:50:15.888216 14816 raft_consensus.cc:388] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:15.888309 14816 raft_consensus.cc:482] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:15.888407 14816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:15.889336 14816 raft_consensus.cc:504] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34490aa2254476ab0e07239dcfbc736" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45853 } }
I20260926 13:50:15.889690 14816 leader_election.cc:302] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f34490aa2254476ab0e07239dcfbc736; no voters:
I20260926 13:50:15.890055 14816 leader_election.cc:288] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:15.890169 14821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:15.890383 14821 raft_consensus.cc:686] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [term 1 LEADER]: Becoming Leader. State: Replica: f34490aa2254476ab0e07239dcfbc736, State: Running, Role: LEADER
I20260926 13:50:15.890748 14821 consensus_queue.cc:237] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34490aa2254476ab0e07239dcfbc736" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45853 } }
I20260926 13:50:15.891467 14816 sys_catalog.cc:564] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:15.892590 14823 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f34490aa2254476ab0e07239dcfbc736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34490aa2254476ab0e07239dcfbc736" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45853 } } }
I20260926 13:50:15.892724 14823 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:15.893119 14822 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f34490aa2254476ab0e07239dcfbc736. Latest consensus state: current_term: 1 leader_uuid: "f34490aa2254476ab0e07239dcfbc736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34490aa2254476ab0e07239dcfbc736" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45853 } } }
I20260926 13:50:15.893251 14822 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:15.893556 14830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:15.895031 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:15.879655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.10.236.254:45853)
0926 13:50:15.880078 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:15.880117 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:15.880162 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:50:15.880814 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:15.880832 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:15.881226 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:15.881532 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:15.882271 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:15.883739 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:15.884659 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:15.885378 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:15.885544 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:15.888309 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:15.888318 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:15.888343 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:15.888406 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:15.890806 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:15.891326 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:15.891333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:15.891339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:15.891513 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:15.891819 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:15.891823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:15.891824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:15.892451 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:15.892671 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:15.893687 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0926 13:50:15.894017 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":145,"threads_started":1}
I20260926 13:50:15.897017 14830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:15.903028 14830 catalog_manager.cc:1347] Generated new cluster ID: be1885a70faa4c2eb5240e54704fdb66
I20260926 13:50:15.903103 14830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:15.910869 14830 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:15.912201 14830 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:15.920594 14830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f34490aa2254476ab0e07239dcfbc736: Generated new TSK 0
I20260926 13:50:15.921451 14830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:15.954912 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45853
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:16.079830 14842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:16.080186 14842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:16.080233 14842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:16.084515 14842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:16.084580 14842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:16.084676 14842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:16.090518 14842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:16.091779 14842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:16.093000 14842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:16.102905 14847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:16.102906 14850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:16.102958 14848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:16.103370 14842 server_base.cc:1034] running on GCE node
I20260926 13:50:16.103909 14842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:16.104593 14842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:16.105783 14842 hybrid_clock.cc:648] HybridClock initialized: now 1790430616105764 us; error 31 us; skew 500 ppm
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:16.109028 14842 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:16.109942 14842 webserver.cc:460] Webserver started at http://127.10.236.193:43807/ using document root <none> and password file <none>
I20260926 13:50:16.110487 14842 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:16.110554 14842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:16.110762 14842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:16.113318 14842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "e097684cf131433bba37bc20d2717372"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "e8203e8fed637cf19589b3f985783b0b"
server_key_iv: "08c6da58ffe0393c05fad95396e32bd5"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.113987 14842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "e097684cf131433bba37bc20d2717372"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "e8203e8fed637cf19589b3f985783b0b"
server_key_iv: "08c6da58ffe0393c05fad95396e32bd5"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.118011 14842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:16.120653 14857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.121855 14842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:16.121995 14842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "e097684cf131433bba37bc20d2717372"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "e8203e8fed637cf19589b3f985783b0b"
server_key_iv: "08c6da58ffe0393c05fad95396e32bd5"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.122113 14842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:16.131908 14842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:16.135372 14842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:16.135613 14842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:16.136245 14842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:16.137080 14842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:16.137159 14842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.137230 14842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:16.137265 14842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.146668 14842 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:43773
I20260926 13:50:16.146692 14970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:43773 every 8 connection(s)
I20260926 13:50:16.147812 14842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:16.152915 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14842
I20260926 13:50:16.153056 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:16.153409 11187 external_mini_cluster.cc:1428] Setting key c20a14a5c74956dbbfa399d3af521121
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:50:16.163712 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.149826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:43779 (local address 127.10.236.254:45853)
0926 13:50:16.150014 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:16.150018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:16.150949 (+ 931us) server_negotiation.cc:407] Connection header received
0926 13:50:16.152502 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:16.152505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:16.152567 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:16.152713 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:16.154099 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.154707 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.156014 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.156438 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.156552 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.159661 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:16.159665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:16.159668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:16.159702 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:16.161349 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.161926 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.161930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.161932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.162025 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.162401 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.162404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.162406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.162607 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:16.162713 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:16.163306 (+ 593us) server_negotiation.cc:299] Negotiation successful
0926 13:50:16.163470 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 13:50:16.164758 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.150230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45853 (local address 127.10.236.193:43779)
0926 13:50:16.150778 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:16.150825 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:16.152251 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:16.152912 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:16.152919 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:16.153417 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:16.153962 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:16.153976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.154843 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.154848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:16.155890 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:16.155900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.156656 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.156662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:16.156892 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.157543 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:16.157566 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:16.159424 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:16.161499 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:16.161506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:16.161526 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:16.161807 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:16.162133 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:16.162136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:16.162138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:16.162295 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:16.162725 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:16.162734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:16.163034 (+ 300us) client_negotiation.cc:769] Sending connection context
0926 13:50:16.163251 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 13:50:16.163650 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:50:16.166381 14971 heartbeater.cc:347] Connected to a master server at 127.10.236.254:45853
I20260926 13:50:16.166673 14971 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:16.167194 14971 heartbeater.cc:511] Master 127.10.236.254:45853 requested a full tablet report, sending...
I20260926 13:50:16.169064 14780 ts_manager.cc:194] Registered new tserver with Master: e097684cf131433bba37bc20d2717372 (127.10.236.193:43773)
I20260926 13:50:16.170384 14780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:43779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:16.184016 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45853
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:16.298404 14976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:16.298660 14976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:16.298707 14976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:16.303011 14976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:16.303085 14976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:16.303154 14976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:16.308558 14976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:16.309553 14976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:16.310619 14976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:16.320179 14984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:16.320252 14976 server_base.cc:1034] running on GCE node
W20260926 13:50:16.320195 14981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:16.320197 14982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:16.320803 14976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:16.321460 14976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:16.322662 14976 hybrid_clock.cc:648] HybridClock initialized: now 1790430616322637 us; error 56 us; skew 500 ppm
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:16.325776 14976 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:16.326869 14976 webserver.cc:460] Webserver started at http://127.10.236.194:43211/ using document root <none> and password file <none>
I20260926 13:50:16.327549 14976 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:16.327647 14976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:16.327895 14976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:16.330084 14976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "72dbdc86fd274deaae59b0cf9a0b484f"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "8b9807e921812c1a468882719ee1c4af"
server_key_iv: "eeae049a82119d169e6093767f04e844"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.330691 14976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "72dbdc86fd274deaae59b0cf9a0b484f"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "8b9807e921812c1a468882719ee1c4af"
server_key_iv: "eeae049a82119d169e6093767f04e844"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.334640 14976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:16.337352 14991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.338657 14976 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:16.338826 14976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "72dbdc86fd274deaae59b0cf9a0b484f"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "8b9807e921812c1a468882719ee1c4af"
server_key_iv: "eeae049a82119d169e6093767f04e844"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.338948 14976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:16.361513 14976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:16.365279 14976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:16.365511 14976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:16.366133 14976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:16.366974 14976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:16.367053 14976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.367126 14976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:16.367173 14976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.376334 14976 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:45633
I20260926 13:50:16.376350 15104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:45633 every 8 connection(s)
I20260926 13:50:16.377295 14976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:16.381001 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 14976
I20260926 13:50:16.381135 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:16.381434 11187 external_mini_cluster.cc:1428] Setting key a1b22dc30bab06306ca2a85bb4cbee85
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:16.393517 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.379203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:59053 (local address 127.10.236.254:45853)
0926 13:50:16.379363 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:16.379367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:16.380226 (+ 859us) server_negotiation.cc:407] Connection header received
0926 13:50:16.381902 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:16.381906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:16.381965 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:16.382071 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:16.383266 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.383887 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.385652 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.385873 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.385963 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.389084 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:16.389089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:16.389093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:16.389133 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:16.390891 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.391637 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.391649 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.391654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.391755 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.392114 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.392117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.392118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.392354 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:16.392435 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:16.393147 (+ 712us) server_negotiation.cc:299] Negotiation successful
0926 13:50:16.393274 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:50:16.394611 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.379613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45853 (local address 127.10.236.194:59053)
0926 13:50:16.380065 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:16.380109 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:16.381638 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:16.382266 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:16.382273 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:16.382629 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:16.383115 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:16.383127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.384013 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.384017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:16.385502 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:16.385517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.386040 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.386046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:16.386295 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.387035 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:16.387056 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:16.388890 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:16.391054 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:16.391061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:16.391075 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:16.391415 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:16.391868 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:16.391871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:16.391873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:16.392015 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:16.392502 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:16.392507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:16.392851 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 13:50:16.393045 (+ 194us) client_negotiation.cc:240] Negotiation successful
0926 13:50:16.393496 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260926 13:50:16.395886 15105 heartbeater.cc:347] Connected to a master server at 127.10.236.254:45853
I20260926 13:50:16.396147 15105 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:16.396646 15105 heartbeater.cc:511] Master 127.10.236.254:45853 requested a full tablet report, sending...
I20260926 13:50:16.397755 14780 ts_manager.cc:194] Registered new tserver with Master: 72dbdc86fd274deaae59b0cf9a0b484f (127.10.236.194:45633)
I20260926 13:50:16.398597 14780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:59053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:16.411389 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45853
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:16.540541 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:16.540874 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:16.540961 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:16.545482 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:16.545578 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:16.545671 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:16.551345 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:16.552523 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:16.553584 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:16.563887 15116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:16.563941 15110 server_base.cc:1034] running on GCE node
W20260926 13:50:16.563828 15115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:16.563828 15118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:16.564579 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:16.565275 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:16.566473 15110 hybrid_clock.cc:648] HybridClock initialized: now 1790430616566442 us; error 39 us; skew 500 ppm
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:16.569576 15110 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:16.570544 15110 webserver.cc:460] Webserver started at http://127.10.236.195:43703/ using document root <none> and password file <none>
I20260926 13:50:16.571166 15110 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:16.571266 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:16.571573 15110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:16.573836 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "ae25fcc2d3064b29b0ec5e2f100e9cf2"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "aa705c098415fd4dc1598c5c30852449"
server_key_iv: "30fedd586021a000b8b95b3dc058835a"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.574456 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "ae25fcc2d3064b29b0ec5e2f100e9cf2"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "aa705c098415fd4dc1598c5c30852449"
server_key_iv: "30fedd586021a000b8b95b3dc058835a"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.578590 15110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260926 13:50:16.581621 15125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.583536 15110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:16.583724 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "ae25fcc2d3064b29b0ec5e2f100e9cf2"
format_stamp: "Formatted at 2026-09-26 13:50:16 on dist-test-slave-r34j"
server_key: "aa705c098415fd4dc1598c5c30852449"
server_key_iv: "30fedd586021a000b8b95b3dc058835a"
server_key_version: "encryptionkey@0"
I20260926 13:50:16.583874 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:16.598371 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:16.601810 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:16.602025 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:16.602648 15110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:16.603528 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:16.603605 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.603686 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:16.603731 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:16.613111 15110 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:34401
I20260926 13:50:16.613265 15238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:34401 every 8 connection(s)
I20260926 13:50:16.614077 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:16.618722 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15110
I20260926 13:50:16.618883 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:16.619231 11187 external_mini_cluster.cc:1428] Setting key 805a7623ae3fd767eb73a6761aaf0e63
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:16.629607 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.616149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:32775 (local address 127.10.236.254:45853)
0926 13:50:16.616302 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:16.616306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:16.617445 (+ 1139us) server_negotiation.cc:407] Connection header received
0926 13:50:16.618360 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:16.618364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:16.618415 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:16.618511 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:16.620241 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.621005 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.622451 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.622694 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.622815 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.625575 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:16.625579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:16.625582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:16.625608 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:16.627365 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.627970 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.627976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.627978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.628074 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.628450 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.628453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.628455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.628655 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:16.628766 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:16.629305 (+ 539us) server_negotiation.cc:299] Negotiation successful
0926 13:50:16.629429 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:50:16.630584 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.616520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45853 (local address 127.10.236.195:32775)
0926 13:50:16.617278 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:16.617323 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:16.618145 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:16.618783 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:16.618794 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:16.619340 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:16.620084 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:16.620102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.621147 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.621151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:16.622321 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:16.622331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.622853 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.622857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:16.623048 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.623755 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:16.623774 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:16.625393 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:16.627523 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:16.627556 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:16.627573 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:16.627854 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:16.628185 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:16.628188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:16.628191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:16.628338 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:16.628777 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:16.628786 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:16.629046 (+ 260us) client_negotiation.cc:769] Sending connection context
0926 13:50:16.629245 (+ 199us) client_negotiation.cc:240] Negotiation successful
0926 13:50:16.629605 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":116,"threads_started":1}
I20260926 13:50:16.631897 15239 heartbeater.cc:347] Connected to a master server at 127.10.236.254:45853
I20260926 13:50:16.632107 15239 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:16.632586 15239 heartbeater.cc:511] Master 127.10.236.254:45853 requested a full tablet report, sending...
I20260926 13:50:16.633836 14780 ts_manager.cc:194] Registered new tserver with Master: ae25fcc2d3064b29b0ec5e2f100e9cf2 (127.10.236.195:34401)
I20260926 13:50:16.634524 14780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:32775
I20260926 13:50:16.646291 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:50:16.673694 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.663574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57374 (local address 127.10.236.193:43773)
0926 13:50:16.663897 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:16.663911 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:16.663936 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:50:16.664036 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:16.664041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:16.664268 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:16.664388 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:16.665091 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.665879 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.666849 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.667247 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.667333 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.670079 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:16.670088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:16.670102 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:16.670127 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:16.671809 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.672317 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.672322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.672323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.672404 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.672666 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.672668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.672669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.672908 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:16.673070 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:16.673340 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 13:50:16.673464 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":109,"threads_started":1}
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[14735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:16.686237 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:16.676937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.10.236.254:45853)
0926 13:50:16.677075 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:16.677080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:16.677196 (+ 116us) server_negotiation.cc:407] Connection header received
0926 13:50:16.677348 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:16.677351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:16.677396 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:16.677491 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:16.678087 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.678615 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.679408 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:16.679749 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:16.679862 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:16.682301 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:16.682305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:16.682307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:16.682341 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:16.683955 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.684442 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.684447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.684449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.684554 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:16.684911 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:16.684915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:16.684916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:16.685281 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:16.685411 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:16.685835 (+ 424us) server_negotiation.cc:299] Negotiation successful
0926 13:50:16.686009 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 13:50:16.690804 14780 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57200
I20260926 13:50:16.691844 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14842
I20260926 13:50:16.697649 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14976
I20260926 13:50:16.703888 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15110
I20260926 13:50:16.709609 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 14748
2026-09-26T13:50:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2218 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[15251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[15251](info): set up 2 sockets
Sep 26 13:50:16 dist-test-slave-r34j krb5kdc[15251](info): commencing operation
krb5kdc: starting...
W20260926 13:50:17.759739 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:17 dist-test-slave-r34j krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:17.891749 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:35795
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:35795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:18.005213 15264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:18.005518 15264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:18.005564 15264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:18.009739 15264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:18.009814 15264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:18.009833 15264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:18.009850 15264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:18.009866 15264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:18.015434 15264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:35795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:35795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:18.016610 15264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:18.017645 15264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:18.026485 15264 server_base.cc:1034] running on GCE node
W20260926 13:50:18.026444 15270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:18.026434 15269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:18.026423 15272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:18.027763 15264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:18.028929 15264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:18.030154 15264 hybrid_clock.cc:648] HybridClock initialized: now 1790430618030127 us; error 42 us; skew 500 ppm
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:18.039881 15264 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:18.040769 15264 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:18.040881 15264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:18.041142 15264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:18.043406 15264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "11cfb9afd28247caa473872bdfc14735"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "388ebe875f6e45628e43a1267c134e09"
server_key_iv: "a493c896f4ab5368d8a950e8b343425e"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.044106 15264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "11cfb9afd28247caa473872bdfc14735"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "388ebe875f6e45628e43a1267c134e09"
server_key_iv: "a493c896f4ab5368d8a950e8b343425e"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.048192 15264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:50:18.050835 15278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.052250 15264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:18.052404 15264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "11cfb9afd28247caa473872bdfc14735"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "388ebe875f6e45628e43a1267c134e09"
server_key_iv: "a493c896f4ab5368d8a950e8b343425e"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.052523 15264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:18.069635 15264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:18.073227 15264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:18.073455 15264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:18.081605 15264 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:35795
I20260926 13:50:18.081945 15330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:35795 every 8 connection(s)
I20260926 13:50:18.082506 15264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:18.085934 15331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:18.088666 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15264
I20260926 13:50:18.088832 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:18.089152 11187 external_mini_cluster.cc:1428] Setting key 12a494ad75446f48a4698b0c56396423
I20260926 13:50:18.092144 15331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: Bootstrap starting.
I20260926 13:50:18.094867 15331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:18.095712 15331 log.cc:824] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:18.097865 15331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: No bootstrap required, opened a new log
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:18.101998 15331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cfb9afd28247caa473872bdfc14735" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 35795 } }
I20260926 13:50:18.102299 15331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:18.102363 15331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11cfb9afd28247caa473872bdfc14735, State: Initialized, Role: FOLLOWER
I20260926 13:50:18.102838 15331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cfb9afd28247caa473872bdfc14735" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 35795 } }
I20260926 13:50:18.103011 15331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:18.103080 15331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:18.103179 15331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:18.104439 15331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cfb9afd28247caa473872bdfc14735" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 35795 } }
I20260926 13:50:18.104803 15331 leader_election.cc:302] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11cfb9afd28247caa473872bdfc14735; no voters:
I20260926 13:50:18.105146 15331 leader_election.cc:288] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:18.105312 15336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:18.105561 15336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [term 1 LEADER]: Becoming Leader. State: Replica: 11cfb9afd28247caa473872bdfc14735, State: Running, Role: LEADER
I20260926 13:50:18.105805 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.090676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.10.236.254:35795)
0926 13:50:18.091137 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.091158 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.091202 (+ 44us) server_negotiation.cc:407] Connection header received
0926 13:50:18.092015 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.092039 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.092476 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.092834 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:18.093533 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.095014 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.096098 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.096667 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.096816 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.099717 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:18.099724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:18.099742 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:18.099775 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:18.102462 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.102940 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.102946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.102952 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.103108 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.103425 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.103431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.103436 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.103969 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:18.104190 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.104395 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.104697 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
I20260926 13:50:18.105929 15336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cfb9afd28247caa473872bdfc14735" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 35795 } }
I20260926 13:50:18.106423 15331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:18.107116 15337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11cfb9afd28247caa473872bdfc14735. Latest consensus state: current_term: 1 leader_uuid: "11cfb9afd28247caa473872bdfc14735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cfb9afd28247caa473872bdfc14735" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 35795 } } }
I20260926 13:50:18.107244 15337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:18.109112 15338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11cfb9afd28247caa473872bdfc14735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cfb9afd28247caa473872bdfc14735" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 35795 } } }
I20260926 13:50:18.109223 15338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735 [sys.catalog]: This master's current role is: LEADER
W20260926 13:50:18.110886 15351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:50:18.110980 15351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:50:18.111059 15345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:18.111876 15345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:18.117543 15345 catalog_manager.cc:1347] Generated new cluster ID: 68620b01a105491d805e78d786cf236f
I20260926 13:50:18.117940 15345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:18.124549 15345 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:18.125504 15345 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:18.130050 15345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: Generated new TSK 0
I20260926 13:50:18.130769 15345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:18.166553 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:35795
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:18.279703 15357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:18.280021 15357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:18.280117 15357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:18.284364 15357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:18.284456 15357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:18.284550 15357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:18.289893 15357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:35795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:18.291040 15357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:18.292187 15357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:18.301786 15362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:18.301782 15363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:18.301875 15357 server_base.cc:1034] running on GCE node
W20260926 13:50:18.301781 15365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:18.302501 15357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:18.303216 15357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:18.304382 15357 hybrid_clock.cc:648] HybridClock initialized: now 1790430618304346 us; error 48 us; skew 500 ppm
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:18.307384 15357 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:18.308255 15357 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:18.308317 15357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:18.308485 15357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:18.310540 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "a36279a4914c4a99b423de676b1ad40b"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "9e9aaa667a67dc2d427d39326279857a"
server_key_iv: "5bb35a343dfb900406cfbe0a3c80b90d"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.311095 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "a36279a4914c4a99b423de676b1ad40b"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "9e9aaa667a67dc2d427d39326279857a"
server_key_iv: "5bb35a343dfb900406cfbe0a3c80b90d"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.314836 15357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:18.317571 15371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.318850 15357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:18.318953 15357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "a36279a4914c4a99b423de676b1ad40b"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "9e9aaa667a67dc2d427d39326279857a"
server_key_iv: "5bb35a343dfb900406cfbe0a3c80b90d"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.319041 15357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:18.337690 15357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:18.341043 15357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:18.341195 15357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:18.341713 15357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:18.342525 15357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:18.342569 15357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.342608 15357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:18.342659 15357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.351325 15357 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:39005
I20260926 13:50:18.351408 15484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:39005 every 8 connection(s)
I20260926 13:50:18.352046 15357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:18.353559 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15357
I20260926 13:50:18.353704 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:18.354017 11187 external_mini_cluster.cc:1428] Setting key b4b0804c504df607685713184853af50
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:50:18.368311 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.353825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:35475 (local address 127.10.236.254:35795)
0926 13:50:18.353985 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.353990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.356349 (+ 2359us) server_negotiation.cc:407] Connection header received
0926 13:50:18.357715 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.357718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.357775 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.357877 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:18.359044 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.359950 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.361166 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.361611 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.361744 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.364348 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:18.364354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:18.364360 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:18.364393 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:18.366050 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.366585 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.366588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.366589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.366669 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.367001 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.367004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.367006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.367279 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:18.367393 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.368005 (+ 612us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.368138 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:50:18.369146 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.354348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:35795 (local address 127.10.236.193:35475)
0926 13:50:18.356170 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.356229 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.357449 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.358053 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.358060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.358412 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:18.358900 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.358912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.360103 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.360106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.361002 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.361011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.361788 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.361791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.361938 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.362622 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:18.362639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:18.364170 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:18.366161 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:18.366165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:18.366177 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:18.366485 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:18.366757 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:18.366759 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:18.366760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:18.366881 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:18.367391 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:18.367398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:18.367743 (+ 345us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.367960 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.368288 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"thread_start_us":142,"threads_started":1}
I20260926 13:50:18.370656 15485 heartbeater.cc:347] Connected to a master server at 127.10.236.254:35795
I20260926 13:50:18.370951 15485 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:18.371455 15485 heartbeater.cc:511] Master 127.10.236.254:35795 requested a full tablet report, sending...
I20260926 13:50:18.373274 15295 ts_manager.cc:194] Registered new tserver with Master: a36279a4914c4a99b423de676b1ad40b (127.10.236.193:39005)
I20260926 13:50:18.374980 15295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:35475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:18.388756 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:35795
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:18.503942 15490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:18.504238 15490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:18.504285 15490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:18.508630 15490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:18.508703 15490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:18.508800 15490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:18.514528 15490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:35795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:18.515771 15490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:18.516924 15490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:18.526791 15498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:18.526816 15496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:18.526927 15490 server_base.cc:1034] running on GCE node
W20260926 13:50:18.526791 15495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:18.527576 15490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:18.528316 15490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:18.529572 15490 hybrid_clock.cc:648] HybridClock initialized: now 1790430618529554 us; error 35 us; skew 500 ppm
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:18.532541 15490 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:18.533318 15490 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:18.533414 15490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:18.533651 15490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:18.535817 15490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "4b4c803efced4a3e99822be9a24083f7"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "6ddd9d6091521fe12dc7d396a335eb1a"
server_key_iv: "91d16b4a3cf461caa6f5e407cf2afc22"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.536387 15490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "4b4c803efced4a3e99822be9a24083f7"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "6ddd9d6091521fe12dc7d396a335eb1a"
server_key_iv: "91d16b4a3cf461caa6f5e407cf2afc22"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.540660 15490 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260926 13:50:18.543242 15504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.544677 15490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:18.544828 15490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "4b4c803efced4a3e99822be9a24083f7"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "6ddd9d6091521fe12dc7d396a335eb1a"
server_key_iv: "91d16b4a3cf461caa6f5e407cf2afc22"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.544952 15490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:18.568405 15490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:18.571936 15490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:18.572198 15490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:18.572862 15490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:18.573779 15490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:18.573869 15490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.573971 15490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:18.574033 15490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.584547 15490 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:37219
I20260926 13:50:18.584615 15617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:37219 every 8 connection(s)
I20260926 13:50:18.585320 15490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:18.586285 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15490
I20260926 13:50:18.586418 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:18.586755 11187 external_mini_cluster.cc:1428] Setting key 47f7b74abb7835cb07edf9bc891fc130
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:18.604939 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.588235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:37241 (local address 127.10.236.254:35795)
0926 13:50:18.588379 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.588384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.590086 (+ 1702us) server_negotiation.cc:407] Connection header received
0926 13:50:18.591465 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.591473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.591621 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.591716 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:18.592972 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.593509 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.594679 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.594926 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.595042 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.600994 (+ 5952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:18.600997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:18.600999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:18.601027 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:18.602601 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.603178 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.603182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.603192 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.603275 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.603748 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.603753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.603755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.603961 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:18.604074 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.604629 (+ 555us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.604765 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:50:18.605904 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.588150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:35795 (local address 127.10.236.194:37241)
0926 13:50:18.589905 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.589976 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.591205 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.591949 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.591958 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.592326 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:18.592820 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.592833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.593641 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.593646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.594548 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.594557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.595082 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.595086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.595281 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.596012 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:18.596029 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:18.600747 (+ 4718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:18.602745 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:18.602754 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:18.602772 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:18.603063 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:18.603376 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:18.603382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:18.603386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:18.603621 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:18.604071 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:18.604079 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:18.604371 (+ 292us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.604570 (+ 199us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.604969 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":142,"threads_started":1}
I20260926 13:50:18.607461 15618 heartbeater.cc:347] Connected to a master server at 127.10.236.254:35795
I20260926 13:50:18.607816 15618 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:18.608436 15618 heartbeater.cc:511] Master 127.10.236.254:35795 requested a full tablet report, sending...
I20260926 13:50:18.609804 15295 ts_manager.cc:194] Registered new tserver with Master: 4b4c803efced4a3e99822be9a24083f7 (127.10.236.194:37219)
I20260926 13:50:18.610541 15295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:37241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:18.619992 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:35795
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:18.732591 15623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:18.732864 15623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:18.732909 15623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:18.736946 15623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:18.737005 15623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:18.737066 15623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:18.742353 15623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:35795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:18.743384 15623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:18.744395 15623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:18.754724 15623 server_base.cc:1034] running on GCE node
W20260926 13:50:18.754663 15628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:18.754663 15631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:18.754663 15629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:18.755677 15623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:18.756459 15623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:18.757661 15623 hybrid_clock.cc:648] HybridClock initialized: now 1790430618757623 us; error 47 us; skew 500 ppm
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:18.761189 15623 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:18.762133 15623 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:18.762213 15623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:18.762399 15623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:18.764756 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "a0d8b10baf874877af4222c5dbd8c314"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "a5b34ae09f055f56d085d2c47f15378e"
server_key_iv: "91d1ff1ab598ea605d6a0874aa2be843"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.765388 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "a0d8b10baf874877af4222c5dbd8c314"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "a5b34ae09f055f56d085d2c47f15378e"
server_key_iv: "91d1ff1ab598ea605d6a0874aa2be843"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.769717 15623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:18.772477 15637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.773907 15623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:18.774068 15623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "a0d8b10baf874877af4222c5dbd8c314"
format_stamp: "Formatted at 2026-09-26 13:50:18 on dist-test-slave-r34j"
server_key: "a5b34ae09f055f56d085d2c47f15378e"
server_key_iv: "91d1ff1ab598ea605d6a0874aa2be843"
server_key_version: "encryptionkey@0"
I20260926 13:50:18.774199 15623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:18.805629 15623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:18.809329 15623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:18.809543 15623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:18.810142 15623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:18.810981 15623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:18.811060 15623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.811170 15623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:18.811223 15623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:18.820672 15623 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:39925
I20260926 13:50:18.820689 15750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:39925 every 8 connection(s)
I20260926 13:50:18.821425 15623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:18.827247 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15623
I20260926 13:50:18.827365 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:18.827735 11187 external_mini_cluster.cc:1428] Setting key 8f9960cab52f757cfaaff8ee553f1da4
Sep 26 13:50:18 dist-test-slave-r34j krb5kdc[15251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430618, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:18.835587 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.823275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:47037 (local address 127.10.236.254:35795)
0926 13:50:18.823401 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.823405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.824628 (+ 1223us) server_negotiation.cc:407] Connection header received
0926 13:50:18.825541 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.825544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.825604 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.825738 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:18.826884 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.827530 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.828936 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.829230 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.829343 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.831990 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:18.831994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:18.831996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:18.832030 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:18.833504 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.834025 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.834030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.834031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.834112 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.834449 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.834454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.834456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.834638 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:18.834765 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.835281 (+ 516us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.835405 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:50:18.836454 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.823697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:35795 (local address 127.10.236.195:47037)
0926 13:50:18.824476 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.824517 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.825322 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.825909 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.825916 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.826271 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:18.826738 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.826750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.827655 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.827658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.828807 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.828824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.829393 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.829397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.829578 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.830206 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:18.830222 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:18.831827 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:18.833637 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:18.833642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:18.833658 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:18.833919 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:18.834209 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:18.834212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:18.834214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:18.834354 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:18.834752 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:18.834758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:18.835030 (+ 272us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.835240 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.835605 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":102,"threads_started":1}
I20260926 13:50:18.837761 15751 heartbeater.cc:347] Connected to a master server at 127.10.236.254:35795
I20260926 13:50:18.838002 15751 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:18.838440 15751 heartbeater.cc:511] Master 127.10.236.254:35795 requested a full tablet report, sending...
I20260926 13:50:18.839576 15295 ts_manager.cc:194] Registered new tserver with Master: a0d8b10baf874877af4222c5dbd8c314 (127.10.236.195:39925)
I20260926 13:50:18.840350 15295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:47037
I20260926 13:50:18.841960 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:50:18.852090 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.844102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.10.236.254:35795)
0926 13:50:18.844223 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.844227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.844345 (+ 118us) server_negotiation.cc:407] Connection header received
0926 13:50:18.844414 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.844416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.844467 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.844539 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:18.845216 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.845736 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.846687 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.846896 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.846977 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.848570 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:18.848573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:18.848575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:18.848597 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:18.850219 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.850657 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.850659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.850661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.850747 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:18.851062 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:18.851067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:18.851068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:18.851294 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:18.851397 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.851799 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.851911 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:50:18.855518 15295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:18.858959 15295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:18.874577 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.868304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:37219 (local address 127.0.0.1:36538)
0926 13:50:18.868951 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.868965 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.869273 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.870447 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.870450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.870469 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:18.870559 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.870565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.871770 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.871773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.872865 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.872877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.873927 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.873933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.874091 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.874131 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.874247 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.874397 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":29,"thread_start_us":86,"threads_started":1}
I20260926 13:50:18.874594 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.868244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.10.236.195:39925)
0926 13:50:18.868723 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.868739 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.868767 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:50:18.869186 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.869193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.869411 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.869577 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:18.870150 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.871300 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.872978 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.873688 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.873764 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.873891 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.874301 (+ 410us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.874399 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":163,"threads_started":1}
I20260926 13:50:18.874771 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.868210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:39925 (local address 127.0.0.1:44624)
0926 13:50:18.868604 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.868662 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.868844 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.869806 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.869812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.869837 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:18.869998 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.870006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.871445 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.871448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.872842 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.872852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.873975 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.873978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.874105 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.874131 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.874248 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.874655 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":333,"thread_start_us":116,"threads_started":1}
I20260926 13:50:18.874993 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.869294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.10.236.193:39005)
0926 13:50:18.869825 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.869833 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.870050 (+ 217us) server_negotiation.cc:407] Connection header received
0926 13:50:18.870257 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.870262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.870431 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.870536 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:18.870928 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.871999 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.873366 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.874189 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.874332 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.874439 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.874554 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.874803 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":68,"threads_started":1}
I20260926 13:50:18.875041 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.869005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:39005 (local address 127.0.0.1:58088)
0926 13:50:18.869962 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.869976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.870111 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.870698 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.870703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.870724 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:18.870800 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.870805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.872122 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.872126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.873242 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.873251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.874359 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.874361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.874433 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.874452 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.874516 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.874570 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":25,"thread_start_us":111,"threads_started":1}
I20260926 13:50:18.875434 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.869436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.10.236.194:37219)
0926 13:50:18.869946 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.869957 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.869981 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:50:18.870062 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.870069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.870265 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.870449 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:18.870661 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.871620 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.872999 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.873790 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.873913 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.874027 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.875223 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.875324 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":162,"threads_started":1}
I20260926 13:50:18.877853 15685 tablet_service.cc:1467] Processing CreateTablet for tablet fafa4a49553748a0b48076a5a0fab636 (DEFAULT_TABLE table=test-table [id=2215714903af4fcfb290efda11b00c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:18.877846 15419 tablet_service.cc:1467] Processing CreateTablet for tablet fafa4a49553748a0b48076a5a0fab636 (DEFAULT_TABLE table=test-table [id=2215714903af4fcfb290efda11b00c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:18.877847 15552 tablet_service.cc:1467] Processing CreateTablet for tablet fafa4a49553748a0b48076a5a0fab636 (DEFAULT_TABLE table=test-table [id=2215714903af4fcfb290efda11b00c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:18.878981 15419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafa4a49553748a0b48076a5a0fab636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:18.878985 15685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafa4a49553748a0b48076a5a0fab636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:18.878989 15552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafa4a49553748a0b48076a5a0fab636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:18.885524 15768 tablet_bootstrap.cc:492] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Bootstrap starting.
I20260926 13:50:18.886679 15769 tablet_bootstrap.cc:492] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: Bootstrap starting.
I20260926 13:50:18.887041 15770 tablet_bootstrap.cc:492] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Bootstrap starting.
I20260926 13:50:18.888517 15768 tablet_bootstrap.cc:654] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:18.888885 15770 tablet_bootstrap.cc:654] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:18.888935 15769 tablet_bootstrap.cc:654] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:18.889333 15768 log.cc:824] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:18.889760 15769 log.cc:824] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:18.889959 15770 log.cc:824] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:18.891299 15768 tablet_bootstrap.cc:492] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: No bootstrap required, opened a new log
I20260926 13:50:18.891559 15768 ts_tablet_manager.cc:1397] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:50:18.891764 15769 tablet_bootstrap.cc:492] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: No bootstrap required, opened a new log
I20260926 13:50:18.891767 15770 tablet_bootstrap.cc:492] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: No bootstrap required, opened a new log
I20260926 13:50:18.891944 15769 ts_tablet_manager.cc:1397] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 13:50:18.891944 15770 ts_tablet_manager.cc:1397] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:50:18.894843 15768 raft_consensus.cc:348] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.895102 15768 raft_consensus.cc:374] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:18.895156 15768 raft_consensus.cc:729] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4c803efced4a3e99822be9a24083f7, State: Initialized, Role: FOLLOWER
I20260926 13:50:18.895668 15768 consensus_queue.cc:260] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.895887 15770 raft_consensus.cc:348] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.896162 15770 raft_consensus.cc:374] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:18.896224 15770 raft_consensus.cc:729] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d8b10baf874877af4222c5dbd8c314, State: Initialized, Role: FOLLOWER
I20260926 13:50:18.896425 15618 heartbeater.cc:503] Master 127.10.236.254:35795 was elected leader, sending a full tablet report...
I20260926 13:50:18.896350 15769 raft_consensus.cc:348] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.896564 15769 raft_consensus.cc:374] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:18.896607 15769 raft_consensus.cc:729] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a36279a4914c4a99b423de676b1ad40b, State: Initialized, Role: FOLLOWER
I20260926 13:50:18.896682 15768 ts_tablet_manager.cc:1428] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:50:18.896685 15770 consensus_queue.cc:260] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.896975 15769 consensus_queue.cc:260] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.897405 15751 heartbeater.cc:503] Master 127.10.236.254:35795 was elected leader, sending a full tablet report...
I20260926 13:50:18.897646 15770 ts_tablet_manager.cc:1428] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:50:18.897763 15485 heartbeater.cc:503] Master 127.10.236.254:35795 was elected leader, sending a full tablet report...
I20260926 13:50:18.898950 15769 ts_tablet_manager.cc:1428] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:50:18.990264 15774 raft_consensus.cc:482] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:18.990476 15774 raft_consensus.cc:504] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.991845 15774 leader_election.cc:288] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0d8b10baf874877af4222c5dbd8c314 (127.10.236.195:39925), a36279a4914c4a99b423de676b1ad40b (127.10.236.193:39005)
I20260926 13:50:18.995986 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.992052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:39925 (local address 127.10.236.194:45245)
0926 13:50:18.992278 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.992297 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.992403 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.992704 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.992710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.992735 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:18.992881 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.992888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.993867 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.993871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.994738 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.994747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.995456 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.995461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.995646 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.995672 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.995750 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.995817 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":1,"spinlock_wait_cycles":12544}
I20260926 13:50:18.996001 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.992213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:45245 (local address 127.10.236.195:39925)
0926 13:50:18.992376 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.992383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.992403 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:50:18.992549 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.992553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.992607 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.992679 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:18.993004 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.993712 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.994872 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.995304 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.995398 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.995438 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.995783 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.995877 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 13:50:18.996054 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.992247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:39005 (local address 127.10.236.194:60415)
0926 13:50:18.992661 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:18.992677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:18.992785 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:18.993034 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:18.993037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:18.993054 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:18.993159 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.993165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.994016 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.994020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.994789 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:18.994824 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.995651 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.995654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:18.995760 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.995784 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:50:18.995859 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 13:50:18.995922 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"spinlock_wait_cycles":1792,"thread_start_us":138,"threads_started":1}
I20260926 13:50:18.996096 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:18.992261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:60415 (local address 127.10.236.193:39005)
0926 13:50:18.992373 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:18.992377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:18.992777 (+ 400us) server_negotiation.cc:407] Connection header received
0926 13:50:18.992871 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:18.992874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:18.992920 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:18.992993 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:18.993286 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.993909 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.994940 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:18.995531 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:18.995657 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:18.995707 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:18.995892 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 13:50:18.995963 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:50:18.996701 15705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fafa4a49553748a0b48076a5a0fab636" candidate_uuid: "4b4c803efced4a3e99822be9a24083f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d8b10baf874877af4222c5dbd8c314" is_pre_election: true
I20260926 13:50:18.996717 15439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fafa4a49553748a0b48076a5a0fab636" candidate_uuid: "4b4c803efced4a3e99822be9a24083f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a36279a4914c4a99b423de676b1ad40b" is_pre_election: true
I20260926 13:50:18.997020 15439 raft_consensus.cc:2454] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4c803efced4a3e99822be9a24083f7 in term 0.
I20260926 13:50:18.997020 15705 raft_consensus.cc:2454] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4c803efced4a3e99822be9a24083f7 in term 0.
I20260926 13:50:18.997517 15505 leader_election.cc:302] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4c803efced4a3e99822be9a24083f7, a0d8b10baf874877af4222c5dbd8c314; no voters:
I20260926 13:50:18.997884 15774 raft_consensus.cc:2781] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:18.997967 15774 raft_consensus.cc:482] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:18.997999 15774 raft_consensus.cc:3037] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:18.999080 15774 raft_consensus.cc:504] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:18.999461 15774 leader_election.cc:288] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [CANDIDATE]: Term 1 election: Requested vote from peers a0d8b10baf874877af4222c5dbd8c314 (127.10.236.195:39925), a36279a4914c4a99b423de676b1ad40b (127.10.236.193:39005)
I20260926 13:50:18.999884 15705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fafa4a49553748a0b48076a5a0fab636" candidate_uuid: "4b4c803efced4a3e99822be9a24083f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d8b10baf874877af4222c5dbd8c314"
I20260926 13:50:18.999905 15439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fafa4a49553748a0b48076a5a0fab636" candidate_uuid: "4b4c803efced4a3e99822be9a24083f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a36279a4914c4a99b423de676b1ad40b"
I20260926 13:50:19.000036 15439 raft_consensus.cc:3037] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:19.000000 15705 raft_consensus.cc:3037] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:19.001346 15705 raft_consensus.cc:2454] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4c803efced4a3e99822be9a24083f7 in term 1.
I20260926 13:50:19.001363 15439 raft_consensus.cc:2454] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4c803efced4a3e99822be9a24083f7 in term 1.
I20260926 13:50:19.001739 15505 leader_election.cc:302] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4c803efced4a3e99822be9a24083f7, a0d8b10baf874877af4222c5dbd8c314; no voters:
I20260926 13:50:19.001942 15774 raft_consensus.cc:2781] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:19.002233 15774 raft_consensus.cc:686] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b4c803efced4a3e99822be9a24083f7, State: Running, Role: LEADER
I20260926 13:50:19.002642 15774 consensus_queue.cc:237] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } }
I20260926 13:50:19.006035 15295 catalog_manager.cc:5495] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4c803efced4a3e99822be9a24083f7 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "4b4c803efced4a3e99822be9a24083f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b4c803efced4a3e99822be9a24083f7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 37219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:19.033016 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.029031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.10.236.194:37219)
0926 13:50:19.029255 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.029260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.029284 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:50:19.029359 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.029361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.029416 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.029541 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:19.029839 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:19.030395 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:19.031356 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:19.031664 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:19.031786 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:19.032142 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:19.032599 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:19.032745 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.032790 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.032852 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 13:50:19.039330 15439 raft_consensus.cc:1261] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 1 FOLLOWER]: Refusing update from remote peer 4b4c803efced4a3e99822be9a24083f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:19.039331 15705 raft_consensus.cc:1261] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 1 FOLLOWER]: Refusing update from remote peer 4b4c803efced4a3e99822be9a24083f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:19.040112 15774 consensus_queue.cc:1035] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0d8b10baf874877af4222c5dbd8c314" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:19.040354 15778 consensus_queue.cc:1035] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a36279a4914c4a99b423de676b1ad40b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:19.048924 15782 mvcc.cc:205] Tried to move back new op lower bound from 7333603815576875008 to 7333603815437729792. Current Snapshot: MvccSnapshot[applied={T|T < 7333603815576875008}]
I20260926 13:50:19.050582 15783 mvcc.cc:205] Tried to move back new op lower bound from 7333603815576875008 to 7333603815437729792. Current Snapshot: MvccSnapshot[applied={T|T < 7333603815576875008 or (T in {7333603815576875008})}]
I20260926 13:50:19.058668 15295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:19.058863 15295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:19.061664 15295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11cfb9afd28247caa473872bdfc14735: Sending DeleteTablet for 3 replicas of tablet fafa4a49553748a0b48076a5a0fab636
I20260926 13:50:19.062688 15419 tablet_service.cc:1514] Processing DeleteTablet for tablet fafa4a49553748a0b48076a5a0fab636 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:19 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:58088
I20260926 13:50:19.062706 15685 tablet_service.cc:1514] Processing DeleteTablet for tablet fafa4a49553748a0b48076a5a0fab636 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:19 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:44624
I20260926 13:50:19.062829 15552 tablet_service.cc:1514] Processing DeleteTablet for tablet fafa4a49553748a0b48076a5a0fab636 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:19 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:36538
I20260926 13:50:19.063288 15791 tablet_replica.cc:330] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b: stopping tablet replica
I20260926 13:50:19.063504 15792 tablet_replica.cc:330] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: stopping tablet replica
I20260926 13:50:19.063705 15791 raft_consensus.cc:2229] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:19.063779 15792 raft_consensus.cc:2229] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:19.063937 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15357
I20260926 13:50:19.064217 15791 raft_consensus.cc:2258] T fafa4a49553748a0b48076a5a0fab636 P a36279a4914c4a99b423de676b1ad40b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:19.064579 15793 tablet_replica.cc:330] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: stopping tablet replica
I20260926 13:50:19.064867 15793 raft_consensus.cc:2229] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:19.065199 15793 raft_consensus.cc:2258] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:19.065580 15792 raft_consensus.cc:2258] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:19.067198 15792 ts_tablet_manager.cc:1905] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:19.067198 15793 ts_tablet_manager.cc:1905] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:19.070144 15792 ts_tablet_manager.cc:1918] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:19.070144 15793 ts_tablet_manager.cc:1918] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:19.070268 15793 log.cc:1196] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/fafa4a49553748a0b48076a5a0fab636
I20260926 13:50:19.070268 15792 log.cc:1196] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/fafa4a49553748a0b48076a5a0fab636
I20260926 13:50:19.070627 15793 ts_tablet_manager.cc:1939] T fafa4a49553748a0b48076a5a0fab636 P 4b4c803efced4a3e99822be9a24083f7: Deleting consensus metadata
I20260926 13:50:19.070627 15792 ts_tablet_manager.cc:1939] T fafa4a49553748a0b48076a5a0fab636 P a0d8b10baf874877af4222c5dbd8c314: Deleting consensus metadata
I20260926 13:50:19.072032 15281 catalog_manager.cc:4841] TS 4b4c803efced4a3e99822be9a24083f7 (127.10.236.194:37219): tablet fafa4a49553748a0b48076a5a0fab636 (table test-table [id=2215714903af4fcfb290efda11b00c61]) successfully deleted
I20260926 13:50:19.072317 15279 catalog_manager.cc:4841] TS a0d8b10baf874877af4222c5dbd8c314 (127.10.236.195:39925): tablet fafa4a49553748a0b48076a5a0fab636 (table test-table [id=2215714903af4fcfb290efda11b00c61]) successfully deleted
W20260926 13:50:19.073881 15281 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:39005 (error 108)
I20260926 13:50:19.074103 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15490
I20260926 13:50:19.076275 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.075915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:39005 (local address 127.0.0.1:58098)
0926 13:50:19.076082 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.076166 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:39005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 13:50:19.076457 15281 catalog_manager.cc:4584] TS a36279a4914c4a99b423de676b1ad40b (127.10.236.193:39005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fafa4a49553748a0b48076a5a0fab636: Network error: Client connection negotiation failed: client connection to 127.10.236.193:39005: connect: Connection refused (error 111)
I20260926 13:50:19.080188 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15623
I20260926 13:50:19.086431 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15264
2026-09-26T13:50:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2376 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T13:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:19Z Disabled control of system clock
I20260926 13:50:19.118034 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:42387
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:42387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 13:50:19.233222 15798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:19.233497 15798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:19.233536 15798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:19.237628 15798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:19.237691 15798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:19.237706 15798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:19.237722 15798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:19.237738 15798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:19.242671 15798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:42387
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:42387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:19.243855 15798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:19.245304 15798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:19.254359 15798 server_base.cc:1034] running on GCE node
W20260926 13:50:19.254287 15806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.254287 15803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.254377 15804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:19.255152 15798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:19.256335 15798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:19.257509 15798 hybrid_clock.cc:648] HybridClock initialized: now 1790430619257470 us; error 53 us; skew 500 ppm
I20260926 13:50:19.259698 15798 webserver.cc:460] Webserver started at http://127.10.236.254:46815/ using document root <none> and password file <none>
I20260926 13:50:19.260293 15798 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:19.260357 15798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:19.260584 15798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:19.262655 15798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "1969b189870648dba6e4e98a9b5b36f3"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "ee4f807803411e01471d15d20fc8cc87"
server_key_iv: "330439a5a3aa79c6bae06e1ec1b1e38c"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.263240 15798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "1969b189870648dba6e4e98a9b5b36f3"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "ee4f807803411e01471d15d20fc8cc87"
server_key_iv: "330439a5a3aa79c6bae06e1ec1b1e38c"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.266928 15798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:19.269526 15812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.270802 15798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:19.270948 15798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "1969b189870648dba6e4e98a9b5b36f3"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "ee4f807803411e01471d15d20fc8cc87"
server_key_iv: "330439a5a3aa79c6bae06e1ec1b1e38c"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.271067 15798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:19.275718 15798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:19.276319 15798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:19.276502 15798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:19.283284 15798 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:42387
I20260926 13:50:19.283295 15864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:42387 every 8 connection(s)
I20260926 13:50:19.284289 15798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:19.285152 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15798
I20260926 13:50:19.285292 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:19.285558 11187 external_mini_cluster.cc:1428] Setting key c465aa52296b342b6d373ff825e2e6ad
I20260926 13:50:19.287703 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:19.292006 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.286947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.10.236.254:42387)
0926 13:50:19.288225 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.288242 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.288277 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:50:19.288967 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.288984 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.289331 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.289677 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.290611 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.290615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.290627 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.290638 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.290728 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.290746 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.290904 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.290975 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.291225 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":731,"threads_started":1}
I20260926 13:50:19.293694 15865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3: Bootstrap starting.
I20260926 13:50:19.295989 15865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:19.296824 15865 log.cc:824] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:19.298749 15865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3: No bootstrap required, opened a new log
I20260926 13:50:19.302297 15865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1969b189870648dba6e4e98a9b5b36f3" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42387 } }
I20260926 13:50:19.302511 15865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:19.302546 15865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1969b189870648dba6e4e98a9b5b36f3, State: Initialized, Role: FOLLOWER
I20260926 13:50:19.302892 15865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1969b189870648dba6e4e98a9b5b36f3" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42387 } }
I20260926 13:50:19.302995 15865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:19.303041 15865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:19.303110 15865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:19.304210 15865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1969b189870648dba6e4e98a9b5b36f3" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42387 } }
I20260926 13:50:19.304498 15865 leader_election.cc:302] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1969b189870648dba6e4e98a9b5b36f3; no voters:
I20260926 13:50:19.304750 15865 leader_election.cc:288] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:19.304876 15870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:19.305106 15870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1969b189870648dba6e4e98a9b5b36f3, State: Running, Role: LEADER
I20260926 13:50:19.305413 15870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1969b189870648dba6e4e98a9b5b36f3" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42387 } }
I20260926 13:50:19.305974 15865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:19.307125 15872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1969b189870648dba6e4e98a9b5b36f3. Latest consensus state: current_term: 1 leader_uuid: "1969b189870648dba6e4e98a9b5b36f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1969b189870648dba6e4e98a9b5b36f3" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42387 } } }
I20260926 13:50:19.307272 15872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:19.307125 15871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1969b189870648dba6e4e98a9b5b36f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1969b189870648dba6e4e98a9b5b36f3" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42387 } } }
I20260926 13:50:19.307374 15871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:19.307705 15879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:19.310442 15879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:19.315928 15879 catalog_manager.cc:1347] Generated new cluster ID: dc88fb37919640559a239c6ae7cb191b
I20260926 13:50:19.315996 15879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:19.336071 15879 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:19.336442 15879 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:19.340142 15879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3: Generated new TSK 0
I20260926 13:50:19.340819 15879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:19.349937 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:42387
--builtin_ntp_servers=127.10.236.212:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 13:50:19.463212 15889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:19.463519 15889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:19.463567 15889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:19.467833 15889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:19.467891 15889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:19.467949 15889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:19.473013 15889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:42387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:19.474248 15889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:19.475790 15889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:19.485504 15889 server_base.cc:1034] running on GCE node
W20260926 13:50:19.485452 15897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.485426 15894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.485464 15895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:19.486047 15889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:19.486708 15889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:19.487895 15889 hybrid_clock.cc:648] HybridClock initialized: now 1790430619487860 us; error 48 us; skew 500 ppm
I20260926 13:50:19.489871 15889 webserver.cc:460] Webserver started at http://127.10.236.193:35605/ using document root <none> and password file <none>
I20260926 13:50:19.490514 15889 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:19.490585 15889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:19.490759 15889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:19.493039 15889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "878d97640c7c40369bc83cab23d4cc47"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "2f1102e38b120ed5f3a52c73821a5c43"
server_key_iv: "5a68d15bec095b13f703b174f7d4640f"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.493582 15889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "878d97640c7c40369bc83cab23d4cc47"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "2f1102e38b120ed5f3a52c73821a5c43"
server_key_iv: "5a68d15bec095b13f703b174f7d4640f"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.497649 15889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:19.500600 15903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.502157 15889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 13:50:19.502305 15889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "878d97640c7c40369bc83cab23d4cc47"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "2f1102e38b120ed5f3a52c73821a5c43"
server_key_iv: "5a68d15bec095b13f703b174f7d4640f"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.502429 15889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:19.507838 15889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:19.508513 15889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:19.508711 15889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:19.509302 15889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:19.510236 15889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:19.510289 15889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.510363 15889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:19.510396 15889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.519289 15889 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:43197
I20260926 13:50:19.519322 16016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:43197 every 8 connection(s)
I20260926 13:50:19.520303 15889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:19.526665 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 15889
I20260926 13:50:19.526796 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:19.527148 11187 external_mini_cluster.cc:1428] Setting key 053b28c9a13824ffd98f0659a8307669
I20260926 13:50:19.527261 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.522410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:55567 (local address 127.10.236.254:42387)
0926 13:50:19.522591 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.522596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.523805 (+ 1209us) server_negotiation.cc:407] Connection header received
0926 13:50:19.525216 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.525220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.525260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.525343 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.526544 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.526549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.526552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.526567 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.526593 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.526604 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.526724 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.526976 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.527096 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 13:50:19.528358 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.522809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:42387 (local address 127.10.236.193:55567)
0926 13:50:19.523624 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.523670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.524971 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.525545 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.525552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.525645 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.526205 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.526207 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.526252 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.526352 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.526709 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.526716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.526720 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.527006 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.527240 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
I20260926 13:50:19.529699 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:42387
--builtin_ntp_servers=127.10.236.212:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 13:50:19.529711 16017 heartbeater.cc:347] Connected to a master server at 127.10.236.254:42387
I20260926 13:50:19.529963 16017 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:19.530396 16017 heartbeater.cc:511] Master 127.10.236.254:42387 requested a full tablet report, sending...
I20260926 13:50:19.532094 15827 ts_manager.cc:194] Registered new tserver with Master: 878d97640c7c40369bc83cab23d4cc47 (127.10.236.193:43197)
W20260926 13:50:19.644424 16020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:19.644738 16020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:19.644820 16020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:19.649013 16020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:19.649096 16020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:19.649190 16020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:19.654090 16020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:42387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:19.655256 16020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:19.656747 16020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:19.666532 16026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.666590 16025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:19.666785 16020 server_base.cc:1034] running on GCE node
W20260926 13:50:19.666532 16028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:19.667331 16020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:19.668030 16020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:19.669232 16020 hybrid_clock.cc:648] HybridClock initialized: now 1790430619669200 us; error 48 us; skew 500 ppm
I20260926 13:50:19.671139 16020 webserver.cc:460] Webserver started at http://127.10.236.194:35497/ using document root <none> and password file <none>
I20260926 13:50:19.671841 16020 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:19.671936 16020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:19.672178 16020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:19.674419 16020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "cdf299e946d944f798216666375388f2"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "a660db4485c676c5a943f0ef30e27b69"
server_key_iv: "840178c436caf7222c04a1d2ae5c8d8e"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.675073 16020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "cdf299e946d944f798216666375388f2"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "a660db4485c676c5a943f0ef30e27b69"
server_key_iv: "840178c436caf7222c04a1d2ae5c8d8e"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.678949 16020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260926 13:50:19.681806 16034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.683091 16020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:19.683243 16020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "cdf299e946d944f798216666375388f2"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "a660db4485c676c5a943f0ef30e27b69"
server_key_iv: "840178c436caf7222c04a1d2ae5c8d8e"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.683367 16020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:19.688313 16020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:19.688949 16020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:19.689137 16020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:19.689674 16020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:19.690605 16020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:19.690675 16020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.690753 16020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:19.690800 16020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.699807 16020 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:36745
I20260926 13:50:19.699846 16147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:36745 every 8 connection(s)
I20260926 13:50:19.700749 16020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:19.706969 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.702607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:52211 (local address 127.10.236.254:42387)
0926 13:50:19.702777 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.702782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.703908 (+ 1126us) server_negotiation.cc:407] Connection header received
0926 13:50:19.705184 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.705186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.705232 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.705310 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.706476 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.706479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.706481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.706492 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.706512 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.706520 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.706598 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.706772 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.706842 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:50:19.707329 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16020
I20260926 13:50:19.707458 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:19.707769 11187 external_mini_cluster.cc:1428] Setting key 8c4af16eafec5cef8369dac51ac85143
I20260926 13:50:19.708133 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.703009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:42387 (local address 127.10.236.194:52211)
0926 13:50:19.703756 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.703799 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.704992 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.705467 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.705474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.705562 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.706200 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.706201 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.706261 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.706330 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.706592 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.706598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.706602 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.706814 (+ 212us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.707114 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":102,"threads_started":1}
I20260926 13:50:19.709311 16148 heartbeater.cc:347] Connected to a master server at 127.10.236.254:42387
I20260926 13:50:19.709535 16148 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:19.709899 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:42387
--builtin_ntp_servers=127.10.236.212:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 13:50:19.709941 16148 heartbeater.cc:511] Master 127.10.236.254:42387 requested a full tablet report, sending...
I20260926 13:50:19.710999 15827 ts_manager.cc:194] Registered new tserver with Master: cdf299e946d944f798216666375388f2 (127.10.236.194:36745)
W20260926 13:50:19.825814 16151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:19.826092 16151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:19.826136 16151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:19.830525 16151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:19.830587 16151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:19.830660 16151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:19.835640 16151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:42387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:19.836853 16151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:19.838477 16151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:19.849352 16156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.849550 16157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:19.849332 16159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:19.849720 16151 server_base.cc:1034] running on GCE node
I20260926 13:50:19.850147 16151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:19.850901 16151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:19.852105 16151 hybrid_clock.cc:648] HybridClock initialized: now 1790430619852049 us; error 71 us; skew 500 ppm
I20260926 13:50:19.854142 16151 webserver.cc:460] Webserver started at http://127.10.236.195:35343/ using document root <none> and password file <none>
I20260926 13:50:19.854758 16151 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:19.854853 16151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:19.855093 16151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:19.857342 16151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "6a2f8727ecbf4ab98b051b84d93d36d4"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "e940b607f4082002df9548b3dfb54f6c"
server_key_iv: "e2ef2253510de9609acf3816b18fdb19"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.857921 16151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "6a2f8727ecbf4ab98b051b84d93d36d4"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "e940b607f4082002df9548b3dfb54f6c"
server_key_iv: "e2ef2253510de9609acf3816b18fdb19"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.862008 16151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:19.864903 16165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.866181 16151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:19.866341 16151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "6a2f8727ecbf4ab98b051b84d93d36d4"
format_stamp: "Formatted at 2026-09-26 13:50:19 on dist-test-slave-r34j"
server_key: "e940b607f4082002df9548b3dfb54f6c"
server_key_iv: "e2ef2253510de9609acf3816b18fdb19"
server_key_version: "encryptionkey@0"
I20260926 13:50:19.866469 16151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:19.871376 16151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:19.872082 16151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:19.872289 16151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:19.872872 16151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:19.873867 16151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:19.873934 16151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.874004 16151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:19.874046 16151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:19.883447 16151 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:45201
I20260926 13:50:19.883467 16278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:45201 every 8 connection(s)
I20260926 13:50:19.884465 16151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:19.886410 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16151
I20260926 13:50:19.886528 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:19.886849 11187 external_mini_cluster.cc:1428] Setting key c36a9c2dde220a28f5bf6299f59f6546
I20260926 13:50:19.891669 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.886673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:36309 (local address 127.10.236.254:42387)
0926 13:50:19.886847 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.886850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.888294 (+ 1444us) server_negotiation.cc:407] Connection header received
0926 13:50:19.889963 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.889965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.890001 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.890077 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.891225 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.891228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.891230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.891241 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.891259 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.891266 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.891337 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.891468 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.891553 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 13:50:19.892650 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.887244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:42387 (local address 127.10.236.195:36309)
0926 13:50:19.888150 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.888207 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.889733 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.890293 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.890301 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.890395 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.890918 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.890920 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.890961 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.891029 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.891344 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.891350 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.891354 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.891545 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.891810 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":169,"threads_started":1}
I20260926 13:50:19.893735 16279 heartbeater.cc:347] Connected to a master server at 127.10.236.254:42387
I20260926 13:50:19.893980 16279 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:19.894423 16279 heartbeater.cc:511] Master 127.10.236.254:42387 requested a full tablet report, sending...
I20260926 13:50:19.895651 15827 ts_manager.cc:194] Registered new tserver with Master: 6a2f8727ecbf4ab98b051b84d93d36d4 (127.10.236.195:45201)
I20260926 13:50:19.901337 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:50:19.905221 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.903527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.10.236.254:42387)
0926 13:50:19.903660 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.903664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.903710 (+ 46us) server_negotiation.cc:407] Connection header received
0926 13:50:19.903947 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.903950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.903988 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.904062 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.904868 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.904871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.904873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.904883 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.904899 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.904907 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.904979 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.905042 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.905076 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:50:19.908541 15827 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:19.912194 15827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:19.925113 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.921518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:45201 (local address 127.0.0.1:56416)
0926 13:50:19.922383 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.922439 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.922569 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.923331 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.923337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.923417 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.924147 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.924151 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.924176 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.924248 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.924729 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.924737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.924743 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.924867 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.924939 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"spinlock_wait_cycles":2048,"thread_start_us":92,"threads_started":1}
I20260926 13:50:19.925184 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.922381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35248 (local address 127.10.236.194:36745)
0926 13:50:19.922824 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.922837 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.922875 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:50:19.923007 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.923013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.923168 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.923283 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.924543 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.924549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.924559 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.924571 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.924617 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.924651 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.924750 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.924964 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.925025 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":139,"threads_started":1}
I20260926 13:50:19.925276 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.921617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:43197 (local address 127.0.0.1:54320)
0926 13:50:19.922439 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.922453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.922569 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.923514 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.923518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.923535 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.924235 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.924237 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.924249 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.924278 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.925043 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.925046 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.925049 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.925114 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.925146 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"spinlock_wait_cycles":21248,"thread_start_us":100,"threads_started":1}
I20260926 13:50:19.925587 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.921943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36745 (local address 127.0.0.1:35248)
0926 13:50:19.922782 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.922809 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.922873 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.923805 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.923808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.923823 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.924389 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.924391 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.924400 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.924422 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.924807 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.924810 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.924813 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.925431 (+ 618us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.925469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":78,"threads_started":1}
I20260926 13:50:19.925729 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.922384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54320 (local address 127.10.236.193:43197)
0926 13:50:19.923052 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.923068 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.923092 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:50:19.923146 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.923152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.923353 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.923526 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.924819 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.924827 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.924841 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.924854 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.924893 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.924916 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.925005 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.925507 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.925575 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":141,"threads_started":1}
I20260926 13:50:19.925161 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.921839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.10.236.195:45201)
0926 13:50:19.922826 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.922843 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.922871 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:50:19.922927 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.922934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.923148 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.923769 (+ 621us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.924442 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.924447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.924457 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.924488 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.924533 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.924603 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.924740 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.924908 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.924996 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":109,"threads_started":1}
I20260926 13:50:19.928537 16082 tablet_service.cc:1467] Processing CreateTablet for tablet cf076ecb442b4cf3857c80d526099c04 (DEFAULT_TABLE table=test-table [id=ddc67875854f478bacd782db29f6194d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:19.928707 15951 tablet_service.cc:1467] Processing CreateTablet for tablet cf076ecb442b4cf3857c80d526099c04 (DEFAULT_TABLE table=test-table [id=ddc67875854f478bacd782db29f6194d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:19.929683 16082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf076ecb442b4cf3857c80d526099c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:19.929797 15951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf076ecb442b4cf3857c80d526099c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:19.929437 16213 tablet_service.cc:1467] Processing CreateTablet for tablet cf076ecb442b4cf3857c80d526099c04 (DEFAULT_TABLE table=test-table [id=ddc67875854f478bacd782db29f6194d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:19.930423 16213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf076ecb442b4cf3857c80d526099c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:19.936043 16296 tablet_bootstrap.cc:492] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Bootstrap starting.
I20260926 13:50:19.937556 16297 tablet_bootstrap.cc:492] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: Bootstrap starting.
I20260926 13:50:19.938633 16298 tablet_bootstrap.cc:492] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Bootstrap starting.
I20260926 13:50:19.939213 16296 tablet_bootstrap.cc:654] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:19.939987 16297 tablet_bootstrap.cc:654] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:19.940582 16296 log.cc:824] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:19.940703 16298 tablet_bootstrap.cc:654] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:19.940829 16297 log.cc:824] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:19.941495 16298 log.cc:824] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:19.942799 16296 tablet_bootstrap.cc:492] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: No bootstrap required, opened a new log
I20260926 13:50:19.943004 16297 tablet_bootstrap.cc:492] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: No bootstrap required, opened a new log
I20260926 13:50:19.943001 16296 ts_tablet_manager.cc:1397] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 13:50:19.943192 16297 ts_tablet_manager.cc:1397] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:50:19.943295 16298 tablet_bootstrap.cc:492] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: No bootstrap required, opened a new log
I20260926 13:50:19.943470 16298 ts_tablet_manager.cc:1397] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:50:19.946969 16298 raft_consensus.cc:348] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.947248 16298 raft_consensus.cc:374] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:19.947309 16298 raft_consensus.cc:729] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a2f8727ecbf4ab98b051b84d93d36d4, State: Initialized, Role: FOLLOWER
I20260926 13:50:19.947803 16298 consensus_queue.cc:260] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.947798 16296 raft_consensus.cc:348] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.947798 16297 raft_consensus.cc:348] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.948043 16297 raft_consensus.cc:374] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:19.948043 16296 raft_consensus.cc:374] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:19.948099 16297 raft_consensus.cc:729] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878d97640c7c40369bc83cab23d4cc47, State: Initialized, Role: FOLLOWER
I20260926 13:50:19.948107 16296 raft_consensus.cc:729] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf299e946d944f798216666375388f2, State: Initialized, Role: FOLLOWER
I20260926 13:50:19.948532 16279 heartbeater.cc:503] Master 127.10.236.254:42387 was elected leader, sending a full tablet report...
I20260926 13:50:19.948576 16296 consensus_queue.cc:260] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.948576 16297 consensus_queue.cc:260] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.949419 16148 heartbeater.cc:503] Master 127.10.236.254:42387 was elected leader, sending a full tablet report...
I20260926 13:50:19.949596 16296 ts_tablet_manager.cc:1428] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 13:50:19.949596 16297 ts_tablet_manager.cc:1428] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 13:50:19.949841 16017 heartbeater.cc:503] Master 127.10.236.254:42387 was elected leader, sending a full tablet report...
W20260926 13:50:19.951676 16149 tablet.cc:2369] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:19.952508 16298 ts_tablet_manager.cc:1428] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260926 13:50:19.974244 16303 raft_consensus.cc:482] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:19.974459 16303 raft_consensus.cc:504] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.975765 16303 leader_election.cc:288] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a2f8727ecbf4ab98b051b84d93d36d4 (127.10.236.195:45201), cdf299e946d944f798216666375388f2 (127.10.236.194:36745)
I20260926 13:50:19.977967 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.976033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36745 (local address 127.10.236.193:55901)
0926 13:50:19.976230 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.976246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.976338 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.976683 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.976687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.976699 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.977229 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.977232 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.977246 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.977278 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.977652 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.977655 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.977659 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.977755 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.977805 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260926 13:50:19.977967 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.976136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:55901 (local address 127.10.236.194:36745)
0926 13:50:19.976282 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.976289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.976303 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:19.976467 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.976470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.976518 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.976638 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.977473 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.977477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.977480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.977495 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.977519 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.977530 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.977659 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.977783 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.977828 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 13:50:19.977967 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.976261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:33247 (local address 127.10.236.195:45201)
0926 13:50:19.976405 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:19.976410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:19.976599 (+ 189us) server_negotiation.cc:407] Connection header received
0926 13:50:19.976697 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:19.976703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:19.976752 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:19.976819 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:19.977473 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:19.977477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:19.977480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:19.977495 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:19.977519 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:19.977530 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:19.977643 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:19.977782 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 13:50:19.977828 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:19.977985 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:19.976059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:45201 (local address 127.10.236.193:33247)
0926 13:50:19.976505 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:19.976516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:19.976585 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:19.976867 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:19.976871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:19.976889 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:19.977290 (+ 401us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:19.977291 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:19.977300 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:19.977322 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:19.977652 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:19.977655 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:19.977659 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:19.977753 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 13:50:19.977816 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":19,"thread_start_us":154,"threads_started":1}
I20260926 13:50:19.978639 16233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf076ecb442b4cf3857c80d526099c04" candidate_uuid: "878d97640c7c40369bc83cab23d4cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" is_pre_election: true
I20260926 13:50:19.978710 16102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf076ecb442b4cf3857c80d526099c04" candidate_uuid: "878d97640c7c40369bc83cab23d4cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdf299e946d944f798216666375388f2" is_pre_election: true
I20260926 13:50:19.978971 16233 raft_consensus.cc:2454] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878d97640c7c40369bc83cab23d4cc47 in term 0.
I20260926 13:50:19.978999 16102 raft_consensus.cc:2454] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878d97640c7c40369bc83cab23d4cc47 in term 0.
I20260926 13:50:19.979557 15905 leader_election.cc:302] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a2f8727ecbf4ab98b051b84d93d36d4, 878d97640c7c40369bc83cab23d4cc47; no voters:
I20260926 13:50:19.979844 16303 raft_consensus.cc:2781] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:19.979952 16303 raft_consensus.cc:482] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:19.980011 16303 raft_consensus.cc:3037] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:19.981061 16303 raft_consensus.cc:504] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.981449 16303 leader_election.cc:288] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [CANDIDATE]: Term 1 election: Requested vote from peers 6a2f8727ecbf4ab98b051b84d93d36d4 (127.10.236.195:45201), cdf299e946d944f798216666375388f2 (127.10.236.194:36745)
I20260926 13:50:19.981860 16102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf076ecb442b4cf3857c80d526099c04" candidate_uuid: "878d97640c7c40369bc83cab23d4cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdf299e946d944f798216666375388f2"
I20260926 13:50:19.981885 16233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf076ecb442b4cf3857c80d526099c04" candidate_uuid: "878d97640c7c40369bc83cab23d4cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4"
I20260926 13:50:19.982008 16233 raft_consensus.cc:3037] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:19.982008 16102 raft_consensus.cc:3037] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:19.983415 16102 raft_consensus.cc:2454] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878d97640c7c40369bc83cab23d4cc47 in term 1.
I20260926 13:50:19.983415 16233 raft_consensus.cc:2454] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878d97640c7c40369bc83cab23d4cc47 in term 1.
I20260926 13:50:19.983786 15904 leader_election.cc:302] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878d97640c7c40369bc83cab23d4cc47, cdf299e946d944f798216666375388f2; no voters:
I20260926 13:50:19.984012 16303 raft_consensus.cc:2781] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:19.984274 16303 raft_consensus.cc:686] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 1 LEADER]: Becoming Leader. State: Replica: 878d97640c7c40369bc83cab23d4cc47, State: Running, Role: LEADER
I20260926 13:50:19.984675 16303 consensus_queue.cc:237] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } }
I20260926 13:50:19.989040 15828 catalog_manager.cc:5495] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878d97640c7c40369bc83cab23d4cc47 (127.10.236.193). New cstate: current_term: 1 leader_uuid: "878d97640c7c40369bc83cab23d4cc47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878d97640c7c40369bc83cab23d4cc47" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 43197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:20.007212 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.005278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54324 (local address 127.10.236.193:43197)
0926 13:50:20.005427 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:20.005433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:20.005503 (+ 70us) server_negotiation.cc:407] Connection header received
0926 13:50:20.005672 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:20.005674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:20.005736 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:20.005846 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:20.006776 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:20.006779 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:20.006782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:20.006796 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:20.006821 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:20.006829 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:20.006924 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:20.007017 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:50:20.007058 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:50:20.014008 16233 raft_consensus.cc:1261] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 1 FOLLOWER]: Refusing update from remote peer 878d97640c7c40369bc83cab23d4cc47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:20.014008 16102 raft_consensus.cc:1261] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 1 FOLLOWER]: Refusing update from remote peer 878d97640c7c40369bc83cab23d4cc47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:20.015131 16306 consensus_queue.cc:1035] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a2f8727ecbf4ab98b051b84d93d36d4" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 45201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:20.015370 16303 consensus_queue.cc:1035] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdf299e946d944f798216666375388f2" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 13:50:20.021577 16018 tablet.cc:2369] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:20.023555 16309 mvcc.cc:205] Tried to move back new op lower bound from 7333603819567075328 to 7333603819460132864. Current Snapshot: MvccSnapshot[applied={T|T < 7333603819567075328}]
I20260926 13:50:20.024098 16311 mvcc.cc:205] Tried to move back new op lower bound from 7333603819567075328 to 7333603819460132864. Current Snapshot: MvccSnapshot[applied={T|T < 7333603819567075328 or (T in {7333603819567075328})}]
I20260926 13:50:20.026219 16310 mvcc.cc:205] Tried to move back new op lower bound from 7333603819567075328 to 7333603819460132864. Current Snapshot: MvccSnapshot[applied={T|T < 7333603819567075328}]
I20260926 13:50:20.032452 15828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:20.032612 15828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:20.035177 15828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1969b189870648dba6e4e98a9b5b36f3: Sending DeleteTablet for 3 replicas of tablet cf076ecb442b4cf3857c80d526099c04
I20260926 13:50:20.036116 15951 tablet_service.cc:1514] Processing DeleteTablet for tablet cf076ecb442b4cf3857c80d526099c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:20 UTC) from {username='slave'} at 127.0.0.1:54320
I20260926 13:50:20.036317 16213 tablet_service.cc:1514] Processing DeleteTablet for tablet cf076ecb442b4cf3857c80d526099c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:20 UTC) from {username='slave'} at 127.0.0.1:56416
I20260926 13:50:20.036495 16082 tablet_service.cc:1514] Processing DeleteTablet for tablet cf076ecb442b4cf3857c80d526099c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:20 UTC) from {username='slave'} at 127.0.0.1:35248
I20260926 13:50:20.036773 16319 tablet_replica.cc:330] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47: stopping tablet replica
I20260926 13:50:20.036815 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15889
I20260926 13:50:20.037104 16319 raft_consensus.cc:2229] T cf076ecb442b4cf3857c80d526099c04 P 878d97640c7c40369bc83cab23d4cc47 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:20.037423 16320 tablet_replica.cc:330] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: stopping tablet replica
I20260926 13:50:20.037503 16321 tablet_replica.cc:330] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: stopping tablet replica
I20260926 13:50:20.037747 16321 raft_consensus.cc:2229] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:20.037747 16320 raft_consensus.cc:2229] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:20.038100 16321 raft_consensus.cc:2258] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:20.038090 16320 raft_consensus.cc:2258] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:20.039915 16320 ts_tablet_manager.cc:1905] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:20.039915 16321 ts_tablet_manager.cc:1905] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:20.042879 16320 ts_tablet_manager.cc:1918] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:20.042879 16321 ts_tablet_manager.cc:1918] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:20.042999 16321 log.cc:1196] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/cf076ecb442b4cf3857c80d526099c04
I20260926 13:50:20.042999 16320 log.cc:1196] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/cf076ecb442b4cf3857c80d526099c04
I20260926 13:50:20.043387 16320 ts_tablet_manager.cc:1939] T cf076ecb442b4cf3857c80d526099c04 P 6a2f8727ecbf4ab98b051b84d93d36d4: Deleting consensus metadata
I20260926 13:50:20.043394 16321 ts_tablet_manager.cc:1939] T cf076ecb442b4cf3857c80d526099c04 P cdf299e946d944f798216666375388f2: Deleting consensus metadata
W20260926 13:50:20.044198 15815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:43197 (error 108)
I20260926 13:50:20.044844 15813 catalog_manager.cc:4841] TS cdf299e946d944f798216666375388f2 (127.10.236.194:36745): tablet cf076ecb442b4cf3857c80d526099c04 (table test-table [id=ddc67875854f478bacd782db29f6194d]) successfully deleted
I20260926 13:50:20.045281 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.044956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:43197 (local address 127.0.0.1:54338)
0926 13:50:20.045088 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:20.045183 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 13:50:20.045398 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16020
I20260926 13:50:20.044844 15814 catalog_manager.cc:4841] TS 6a2f8727ecbf4ab98b051b84d93d36d4 (127.10.236.195:45201): tablet cf076ecb442b4cf3857c80d526099c04 (table test-table [id=ddc67875854f478bacd782db29f6194d]) successfully deleted
W20260926 13:50:20.045496 15815 catalog_manager.cc:4584] TS 878d97640c7c40369bc83cab23d4cc47 (127.10.236.193:43197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf076ecb442b4cf3857c80d526099c04: Network error: Client connection negotiation failed: client connection to 127.10.236.193:43197: connect: Connection refused (error 111)
I20260926 13:50:20.051314 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16151
I20260926 13:50:20.057420 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 15798
2026-09-26T13:50:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (969 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 13:50:20.078966 11187 mini_oidc.cc:152] Starting JWKS server
I20260926 13:50:20.080652 11187 webserver.cc:460] Webserver started at https://127.0.0.1:33521/ using document root <none> and password file <none>
I20260926 13:50:20.080799 11187 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 13:50:20.081028 11187 webserver.cc:460] Webserver started at http://127.0.0.1:33587/ using document root <none> and password file <none>
2026-09-26T13:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:20Z Disabled control of system clock
I20260926 13:50:20.090814 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:38625
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:45007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:38625
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33521/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/testchainca.cert with env {}
W20260926 13:50:20.204298 16328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:20.204622 16328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:20.204696 16328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:20.208827 16328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:20.208923 16328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:20.208951 16328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:20.208992 16328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:20.209022 16328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:50:20.209071 16328 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:50:20.209107 16328 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33521/jwks.json
I20260926 13:50:20.214578 16328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:45007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:38625
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:38625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33521/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:20.215791 16328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:20.216818 16328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:20.225417 16336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:20.225461 16333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:20.225417 16334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:20.225615 16328 server_base.cc:1034] running on GCE node
I20260926 13:50:20.226241 16328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:20.227268 16328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:20.228428 16328 hybrid_clock.cc:648] HybridClock initialized: now 1790430620228414 us; error 49 us; skew 500 ppm
I20260926 13:50:20.230329 16328 webserver.cc:460] Webserver started at http://127.10.236.254:34619/ using document root <none> and password file <none>
I20260926 13:50:20.230916 16328 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:20.230976 16328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:20.231189 16328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:20.233290 16328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "a91a8e253b7c463abbe5e736debc3dfa"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "5035a2411fbb05661a6067da91f6a66d"
server_key_iv: "449995d446733825aea982467e860305"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.233856 16328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "a91a8e253b7c463abbe5e736debc3dfa"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "5035a2411fbb05661a6067da91f6a66d"
server_key_iv: "449995d446733825aea982467e860305"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.237488 16328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:50:20.240110 16342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:20.241436 16328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:20.241580 16328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "a91a8e253b7c463abbe5e736debc3dfa"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "5035a2411fbb05661a6067da91f6a66d"
server_key_iv: "449995d446733825aea982467e860305"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.241700 16328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:20.287734 16328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:20.288492 16328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:20.288715 16328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:20.296114 16328 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:38625
I20260926 13:50:20.296121 16394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:38625 every 8 connection(s)
I20260926 13:50:20.297093 16328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:20.297953 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16328
I20260926 13:50:20.298090 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:20.298413 11187 external_mini_cluster.cc:1428] Setting key 7a1f886b35912f4c304a4df0bbdc8c47
I20260926 13:50:20.301016 16395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:20.307070 16395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa: Bootstrap starting.
I20260926 13:50:20.307750 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.299850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.10.236.254:38625)
0926 13:50:20.300472 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:20.300488 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:20.300522 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:50:20.301247 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:20.301266 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:20.301650 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:20.301999 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:20.302424 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.303385 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.304306 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.304784 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.304974 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.305910 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:20.305914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:20.305927 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:20.306319 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:20.306374 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:20.306393 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:20.306503 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:20.306656 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 13:50:20.306920 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":130,"threads_started":1}
I20260926 13:50:20.309561 16395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:20.310564 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:50:20.310573 16395 log.cc:824] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:20.312701 16395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa: No bootstrap required, opened a new log
I20260926 13:50:20.316282 16395 raft_consensus.cc:348] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91a8e253b7c463abbe5e736debc3dfa" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38625 } }
I20260926 13:50:20.316509 16395 raft_consensus.cc:374] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:20.316591 16395 raft_consensus.cc:729] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a91a8e253b7c463abbe5e736debc3dfa, State: Initialized, Role: FOLLOWER
I20260926 13:50:20.316995 16395 consensus_queue.cc:260] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91a8e253b7c463abbe5e736debc3dfa" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38625 } }
I20260926 13:50:20.317138 16395 raft_consensus.cc:388] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:20.317201 16395 raft_consensus.cc:482] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:20.317312 16395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:20.318341 16395 raft_consensus.cc:504] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91a8e253b7c463abbe5e736debc3dfa" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38625 } }
I20260926 13:50:20.318672 16395 leader_election.cc:302] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a91a8e253b7c463abbe5e736debc3dfa; no voters:
I20260926 13:50:20.318974 16395 leader_election.cc:288] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:20.319132 16400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:20.319404 16400 raft_consensus.cc:686] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [term 1 LEADER]: Becoming Leader. State: Replica: a91a8e253b7c463abbe5e736debc3dfa, State: Running, Role: LEADER
I20260926 13:50:20.319725 16400 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91a8e253b7c463abbe5e736debc3dfa" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38625 } }
I20260926 13:50:20.320398 16395 sys_catalog.cc:564] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:20.321408 16402 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91a8e253b7c463abbe5e736debc3dfa. Latest consensus state: current_term: 1 leader_uuid: "a91a8e253b7c463abbe5e736debc3dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91a8e253b7c463abbe5e736debc3dfa" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38625 } } }
I20260926 13:50:20.321430 16401 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a91a8e253b7c463abbe5e736debc3dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91a8e253b7c463abbe5e736debc3dfa" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38625 } } }
I20260926 13:50:20.321667 16401 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:20.321692 16402 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:20.322080 16409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:20.326051 16409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:20.332587 16409 catalog_manager.cc:1347] Generated new cluster ID: ba1918d1a30044e8b8f1b682ad2427ae
I20260926 13:50:20.332674 16409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:20.355115 16409 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:20.356442 16409 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:20.370095 16409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a91a8e253b7c463abbe5e736debc3dfa: Generated new TSK 0
I20260926 13:50:20.370959 16409 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:50:20.413085 16424 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:20.385893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38625 (local address 127.0.0.1:51634)
0926 13:50:20.386250 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:20.386266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:20.386398 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:20.386728 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:20.386731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:20.386740 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:50:20.386847 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:20.386854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.387683 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.387687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:20.388676 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:20.388686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.389185 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.389189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:20.389285 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.389336 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:50:20.411425 (+ 22089us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 13:50:20.411704 (+ 279us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.236.254:38625: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
I20260926 13:50:20.413673 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.385983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.10.236.254:38625)
0926 13:50:20.386136 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:20.386143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:20.386341 (+ 198us) server_negotiation.cc:407] Connection header received
0926 13:50:20.386547 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:20.386549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:20.386594 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:20.386680 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:50:20.386950 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.387567 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.388831 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.389058 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.389159 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.389517 (+ 358us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:50:20.411110 (+ 21593us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 13:50:20.413532 (+ 2422us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51634: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 13:50:20.413740 16398 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51634: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.236.254:38625: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33521/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 13:50:20.414304 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16328
E20260926 13:50:20.414369 16425 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 13:50:20.415012 16425 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T13:50:20Z chronyd exiting
I20260926 13:50:20.452432 11187 test_util.cc:175] -----------------------------------------------
I20260926 13:50:20.452541 11187 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790430594225530-11187-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (375 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 13:50:20.453604 11187 mini_oidc.cc:152] Starting JWKS server
I20260926 13:50:20.454474 11187 webserver.cc:460] Webserver started at https://127.0.0.1:43841/ using document root <none> and password file <none>
I20260926 13:50:20.454603 11187 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 13:50:20.454900 11187 webserver.cc:460] Webserver started at http://127.0.0.1:37663/ using document root <none> and password file <none>
2026-09-26T13:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:20Z Disabled control of system clock
I20260926 13:50:20.464110 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:42707
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:42707
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43841/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/testchainca.cert with env {}
W20260926 13:50:20.576390 16431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:20.576648 16431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:20.576689 16431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:20.581010 16431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:20.581104 16431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:20.581130 16431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:20.581154 16431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:20.581187 16431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:50:20.581229 16431 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:50:20.581256 16431 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43841/jwks.json
I20260926 13:50:20.588959 16431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:42707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:42707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43841/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:20.590099 16431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:20.591127 16431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:20.599795 16436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:20.599851 16431 server_base.cc:1034] running on GCE node
W20260926 13:50:20.599807 16439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:20.599812 16437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:20.600611 16431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:20.601761 16431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:20.602926 16431 hybrid_clock.cc:648] HybridClock initialized: now 1790430620602894 us; error 49 us; skew 500 ppm
I20260926 13:50:20.604809 16431 webserver.cc:460] Webserver started at http://127.10.236.254:38019/ using document root <none> and password file <none>
I20260926 13:50:20.605386 16431 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:20.605448 16431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:20.605701 16431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:20.607796 16431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "5ca5b55f646145e2929a7d419842e4a1"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "391f6931fdbe8430ea0c5a28de225e74"
server_key_iv: "20e78b85fb4daa72f4fccc7b02b5074e"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.608350 16431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "5ca5b55f646145e2929a7d419842e4a1"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "391f6931fdbe8430ea0c5a28de225e74"
server_key_iv: "20e78b85fb4daa72f4fccc7b02b5074e"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.612041 16431 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 13:50:20.614544 16445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:20.615964 16431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:20.616114 16431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "5ca5b55f646145e2929a7d419842e4a1"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "391f6931fdbe8430ea0c5a28de225e74"
server_key_iv: "20e78b85fb4daa72f4fccc7b02b5074e"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.616246 16431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:20.636724 16431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:20.637486 16431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:20.637701 16431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:20.645077 16431 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:42707
I20260926 13:50:20.645251 16497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:42707 every 8 connection(s)
I20260926 13:50:20.646121 16431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:20.649499 16498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:20.650286 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16431
I20260926 13:50:20.650411 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:20.650710 11187 external_mini_cluster.cc:1428] Setting key 1335431bd794ae1ac0267002f408745e
I20260926 13:50:20.655846 16498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1: Bootstrap starting.
I20260926 13:50:20.658186 16498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:20.658914 16498 log.cc:824] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:20.660568 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.652190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.10.236.254:42707)
0926 13:50:20.652861 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:20.652883 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:20.652920 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:50:20.653563 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:20.653581 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:20.653979 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:20.654385 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:20.654881 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.656039 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.656911 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.657392 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.657579 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.658509 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:20.658518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:20.658535 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:20.658924 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:20.658977 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:20.658995 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:20.659107 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:20.659316 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 13:50:20.659620 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":132,"threads_started":1}
I20260926 13:50:20.661176 16498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1: No bootstrap required, opened a new log
I20260926 13:50:20.663431 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:50:20.665285 16498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca5b55f646145e2929a7d419842e4a1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42707 } }
I20260926 13:50:20.665515 16498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:20.665597 16498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca5b55f646145e2929a7d419842e4a1, State: Initialized, Role: FOLLOWER
I20260926 13:50:20.666060 16498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca5b55f646145e2929a7d419842e4a1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42707 } }
I20260926 13:50:20.666208 16498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:20.666306 16498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:20.666426 16498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:20.667333 16498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca5b55f646145e2929a7d419842e4a1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42707 } }
I20260926 13:50:20.667713 16498 leader_election.cc:302] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ca5b55f646145e2929a7d419842e4a1; no voters:
I20260926 13:50:20.668043 16498 leader_election.cc:288] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:20.668116 16503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:20.668421 16503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ca5b55f646145e2929a7d419842e4a1, State: Running, Role: LEADER
I20260926 13:50:20.668754 16503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca5b55f646145e2929a7d419842e4a1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42707 } }
I20260926 13:50:20.669485 16498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:20.670482 16505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ca5b55f646145e2929a7d419842e4a1. Latest consensus state: current_term: 1 leader_uuid: "5ca5b55f646145e2929a7d419842e4a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca5b55f646145e2929a7d419842e4a1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42707 } } }
I20260926 13:50:20.670482 16504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ca5b55f646145e2929a7d419842e4a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca5b55f646145e2929a7d419842e4a1" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42707 } } }
I20260926 13:50:20.670639 16505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:20.670639 16504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:20.671077 16512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:20.674307 16512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:20.679899 16512 catalog_manager.cc:1347] Generated new cluster ID: f21cd42761d747ef813cbeff464d0848
I20260926 13:50:20.679978 16512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:20.689138 16512 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:20.690078 16512 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:20.700995 16512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ca5b55f646145e2929a7d419842e4a1: Generated new TSK 0
I20260926 13:50:20.701699 16512 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 13:50:20.731005 16528 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 13:50:20.731108 16528 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 13:50:20.731890 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.715005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.10.236.254:42707)
0926 13:50:20.715143 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:20.715149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:20.715298 (+ 149us) server_negotiation.cc:407] Connection header received
0926 13:50:20.715441 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:20.715445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:20.715525 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:20.715651 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:50:20.715861 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.716691 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.717517 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.717867 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.717968 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.718224 (+ 256us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:50:20.731422 (+ 13198us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 13:50:20.731748 (+ 326us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55332: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 13:50:20.731957 16501 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55332: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 13:50:20.731997 16527 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:20.714866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:42707 (local address 127.0.0.1:55332)
0926 13:50:20.715208 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:20.715220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:20.715331 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:20.715647 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:20.715649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:20.715656 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:50:20.715758 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:20.715764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.716796 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.716798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:20.717405 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:20.717411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.717977 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.717980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:20.718040 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.718060 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:50:20.731666 (+ 13606us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 13:50:20.731823 (+ 157us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.236.254:42707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260926 13:50:20.732668 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16431
2026-09-26T13:50:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (318 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 13:50:20.772151 11187 mini_oidc.cc:152] Starting JWKS server
I20260926 13:50:20.773022 11187 webserver.cc:460] Webserver started at https://127.0.0.1:41395/ using document root <none> and password file <none>
I20260926 13:50:20.773162 11187 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 13:50:20.773428 11187 webserver.cc:460] Webserver started at http://127.0.0.1:37617/ using document root <none> and password file <none>
2026-09-26T13:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:20Z Disabled control of system clock
I20260926 13:50:20.783098 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44475
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:36859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44475
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41395/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 13:50:20.892910 16534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:20.893193 16534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:20.893234 16534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:20.897850 16534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:20.897918 16534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:20.897933 16534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:20.897951 16534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:20.897966 16534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:50:20.897982 16534 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:50:20.897996 16534 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41395/jwks.json
I20260926 13:50:20.902925 16534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:36859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41395/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:20.904083 16534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:20.905448 16534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:20.914194 16540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:20.914196 16542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:20.914428 16534 server_base.cc:1034] running on GCE node
W20260926 13:50:20.914213 16539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:20.915133 16534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:20.916302 16534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:20.917464 16534 hybrid_clock.cc:648] HybridClock initialized: now 1790430620917455 us; error 37 us; skew 500 ppm
I20260926 13:50:20.919584 16534 webserver.cc:460] Webserver started at http://127.10.236.254:41747/ using document root <none> and password file <none>
I20260926 13:50:20.920158 16534 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:20.920222 16534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:20.920401 16534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:20.922526 16534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "1858a48cb5724ce296816e5dfb5aae76"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "956ca8588988fc9e5cc1aa23ee3fb88c"
server_key_iv: "b3d1175704e6c6b23794cdcd3c7d525d"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.923058 16534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "1858a48cb5724ce296816e5dfb5aae76"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "956ca8588988fc9e5cc1aa23ee3fb88c"
server_key_iv: "b3d1175704e6c6b23794cdcd3c7d525d"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.926800 16534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260926 13:50:20.929275 16548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:20.930531 16534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:20.930632 16534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "1858a48cb5724ce296816e5dfb5aae76"
format_stamp: "Formatted at 2026-09-26 13:50:20 on dist-test-slave-r34j"
server_key: "956ca8588988fc9e5cc1aa23ee3fb88c"
server_key_iv: "b3d1175704e6c6b23794cdcd3c7d525d"
server_key_version: "encryptionkey@0"
I20260926 13:50:20.930724 16534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:20.944085 16534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:20.944736 16534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:20.944887 16534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:20.951798 16534 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44475
I20260926 13:50:20.951865 16600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44475 every 8 connection(s)
I20260926 13:50:20.952816 16534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:20.955943 16601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:20.959350 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16534
I20260926 13:50:20.959514 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:20.959825 11187 external_mini_cluster.cc:1428] Setting key bf468272a3a2d6b476eb8009c41592a6
I20260926 13:50:20.962216 16601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76: Bootstrap starting.
I20260926 13:50:20.964588 16601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:20.965803 16601 log.cc:824] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:20.968564 16601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76: No bootstrap required, opened a new log
I20260926 13:50:20.969923 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:20.961260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.10.236.254:44475)
0926 13:50:20.961941 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:20.961957 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:20.961993 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:50:20.962686 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:20.962706 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:20.963130 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:20.963488 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:20.963991 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.965075 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.966059 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:20.966648 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:20.966788 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:20.967806 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:20.967813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:20.967830 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:20.968334 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:20.968409 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:20.968435 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:20.968539 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:20.968703 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 13:50:20.969001 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":139,"threads_started":1}
I20260926 13:50:20.973006 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:50:20.972990 16601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1858a48cb5724ce296816e5dfb5aae76" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44475 } }
I20260926 13:50:20.973275 16601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:20.973357 16601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1858a48cb5724ce296816e5dfb5aae76, State: Initialized, Role: FOLLOWER
I20260926 13:50:20.973816 16601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1858a48cb5724ce296816e5dfb5aae76" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44475 } }
I20260926 13:50:20.973966 16601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:20.974056 16601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:20.974217 16601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:20.975322 16601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1858a48cb5724ce296816e5dfb5aae76" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44475 } }
I20260926 13:50:20.975710 16601 leader_election.cc:302] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1858a48cb5724ce296816e5dfb5aae76; no voters:
I20260926 13:50:20.976028 16601 leader_election.cc:288] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:20.976217 16606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:20.976449 16606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [term 1 LEADER]: Becoming Leader. State: Replica: 1858a48cb5724ce296816e5dfb5aae76, State: Running, Role: LEADER
I20260926 13:50:20.976778 16606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1858a48cb5724ce296816e5dfb5aae76" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44475 } }
I20260926 13:50:20.977319 16601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:20.978421 16608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1858a48cb5724ce296816e5dfb5aae76. Latest consensus state: current_term: 1 leader_uuid: "1858a48cb5724ce296816e5dfb5aae76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1858a48cb5724ce296816e5dfb5aae76" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44475 } } }
I20260926 13:50:20.978546 16608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:20.978847 16607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1858a48cb5724ce296816e5dfb5aae76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1858a48cb5724ce296816e5dfb5aae76" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44475 } } }
I20260926 13:50:20.978931 16607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:20.978962 16615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:20.982537 16615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:20.988708 16615 catalog_manager.cc:1347] Generated new cluster ID: b45e5a4ab31a4b9b82f551ce3755f981
I20260926 13:50:20.988824 16615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:21.012585 16615 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:21.013690 16615 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:21.020489 16615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1858a48cb5724ce296816e5dfb5aae76: Generated new TSK 0
I20260926 13:50:21.021201 16615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:21.041450 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:21.036009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.10.236.254:44475)
0926 13:50:21.036151 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:21.036156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:21.036407 (+ 251us) server_negotiation.cc:407] Connection header received
0926 13:50:21.036541 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:21.036544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:21.036610 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:21.036708 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:50:21.037007 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.037609 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.038864 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.039095 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.039193 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:21.039637 (+ 444us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:50:21.041087 (+ 1450us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 13:50:21.041314 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 13:50:21.041518 16604 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 13:50:21.043349 16630 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:21.035944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44475 (local address 127.0.0.1:47916)
0926 13:50:21.036280 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:21.036296 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:21.036415 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:21.036735 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:21.036738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:21.036746 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:50:21.036859 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:21.036865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.037740 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.037744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:21.038730 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:21.038739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.039244 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.039248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:21.039335 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:21.039364 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:50:21.041258 (+ 1894us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 13:50:21.043166 (+ 1908us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.236.254:44475: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":1711,"thread_start_us":119,"threads_started":1}
I20260926 13:50:21.044042 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16534
E20260926 13:50:21.054212 16631 webserver.cc:533] Webserver: sslize failed:
E20260926 13:50:21.054316 16631 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T13:50:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (312 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T13:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:21Z Disabled control of system clock
I20260926 13:50:21.093775 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34489
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:34489
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/testchainca.cert with env {}
W20260926 13:50:21.204679 16635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:21.204963 16635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:21.205005 16635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:21.209721 16635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:21.209794 16635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:21.209816 16635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:21.209834 16635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:21.209851 16635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:50:21.209896 16635 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:50:21.209926 16635 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 13:50:21.215726 16635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:34489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:21.216905 16635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:21.217945 16635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:21.226943 16641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:21.226966 16640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:21.226960 16643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:21.227217 16635 server_base.cc:1034] running on GCE node
I20260926 13:50:21.228044 16635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:21.229276 16635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:21.230520 16635 hybrid_clock.cc:648] HybridClock initialized: now 1790430621230504 us; error 38 us; skew 500 ppm
I20260926 13:50:21.232599 16635 webserver.cc:460] Webserver started at http://127.10.236.254:35089/ using document root <none> and password file <none>
I20260926 13:50:21.233223 16635 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:21.233318 16635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:21.233562 16635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:21.235637 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "44f30443abd9420dbff3607b3c151a3b"
format_stamp: "Formatted at 2026-09-26 13:50:21 on dist-test-slave-r34j"
server_key: "4088cd73943b2ef8193fda61beeb4881"
server_key_iv: "f87fca5eadb32a81b7e0cf189fb465f0"
server_key_version: "encryptionkey@0"
I20260926 13:50:21.236214 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "44f30443abd9420dbff3607b3c151a3b"
format_stamp: "Formatted at 2026-09-26 13:50:21 on dist-test-slave-r34j"
server_key: "4088cd73943b2ef8193fda61beeb4881"
server_key_iv: "f87fca5eadb32a81b7e0cf189fb465f0"
server_key_version: "encryptionkey@0"
I20260926 13:50:21.239873 16635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:21.242292 16649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:21.243588 16635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:21.243729 16635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "44f30443abd9420dbff3607b3c151a3b"
format_stamp: "Formatted at 2026-09-26 13:50:21 on dist-test-slave-r34j"
server_key: "4088cd73943b2ef8193fda61beeb4881"
server_key_iv: "f87fca5eadb32a81b7e0cf189fb465f0"
server_key_version: "encryptionkey@0"
I20260926 13:50:21.243860 16635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:21.261818 16635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:21.262552 16635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:21.262769 16635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:21.269568 16635 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:34489
I20260926 13:50:21.269590 16701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:34489 every 8 connection(s)
I20260926 13:50:21.270594 16635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:21.273506 16702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:21.278965 16702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b: Bootstrap starting.
I20260926 13:50:21.280114 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16635
I20260926 13:50:21.280223 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:21.280529 11187 external_mini_cluster.cc:1428] Setting key 6aa2e759be1104d23315f04b94c162ab
I20260926 13:50:21.281806 16702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:21.282699 16702 log.cc:824] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:21.285063 16702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b: No bootstrap required, opened a new log
I20260926 13:50:21.289369 16702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f30443abd9420dbff3607b3c151a3b" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34489 } }
I20260926 13:50:21.289654 16702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:21.289712 16702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f30443abd9420dbff3607b3c151a3b, State: Initialized, Role: FOLLOWER
I20260926 13:50:21.290189 16702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f30443abd9420dbff3607b3c151a3b" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34489 } }
I20260926 13:50:21.290364 16702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:21.290421 16702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:21.290506 16702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:21.290876 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:21.281599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.10.236.254:34489)
0926 13:50:21.282341 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:21.282359 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:21.282403 (+ 44us) server_negotiation.cc:407] Connection header received
0926 13:50:21.283244 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:21.283264 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:21.283712 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:21.284087 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:21.284549 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.285965 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.286900 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.287658 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.287814 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:21.288756 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:21.288762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:21.288782 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:21.289291 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:21.289363 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:21.289392 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:21.289492 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:21.289592 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 13:50:21.289877 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":129,"threads_started":1}
I20260926 13:50:21.291700 16702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f30443abd9420dbff3607b3c151a3b" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34489 } }
I20260926 13:50:21.292053 16702 leader_election.cc:302] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f30443abd9420dbff3607b3c151a3b; no voters:
I20260926 13:50:21.292387 16702 leader_election.cc:288] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:21.292460 16707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:21.292655 16707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [term 1 LEADER]: Becoming Leader. State: Replica: 44f30443abd9420dbff3607b3c151a3b, State: Running, Role: LEADER
I20260926 13:50:21.293249 16707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f30443abd9420dbff3607b3c151a3b" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34489 } }
I20260926 13:50:21.293674 16702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:21.294055 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:50:21.294894 16709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f30443abd9420dbff3607b3c151a3b. Latest consensus state: current_term: 1 leader_uuid: "44f30443abd9420dbff3607b3c151a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f30443abd9420dbff3607b3c151a3b" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34489 } } }
I20260926 13:50:21.295023 16709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:21.295324 16708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f30443abd9420dbff3607b3c151a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f30443abd9420dbff3607b3c151a3b" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34489 } } }
I20260926 13:50:21.295426 16708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:21.296149 16716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:21.299145 16716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:21.304644 16716 catalog_manager.cc:1347] Generated new cluster ID: 44311dcf85fe4fc6aef5ba16719e7c53
I20260926 13:50:21.304713 16716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:21.320284 16716 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:21.321215 16716 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:21.326804 16716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f30443abd9420dbff3607b3c151a3b: Generated new TSK 0
I20260926 13:50:21.327822 16716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:21.867961 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:21.344804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.10.236.254:34489)
0926 13:50:21.344946 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:21.344951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:21.345120 (+ 169us) server_negotiation.cc:407] Connection header received
0926 13:50:21.345287 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:21.345289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:21.345338 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:21.345423 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:50:21.345676 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.346209 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.347541 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.347802 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.347891 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:21.348270 (+ 379us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:50:21.867471 (+519201us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 13:50:21.867841 (+ 370us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43078: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 13:50:21.868064 16705 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43078: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 13:50:21.868256 16731 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:21.344715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34489 (local address 127.0.0.1:43078)
0926 13:50:21.345023 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:21.345041 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:21.345158 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:21.345449 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:21.345452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:21.345460 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:50:21.345573 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:21.345579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.346345 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.346349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:21.347378 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:21.347388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:21.347933 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:21.347937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:21.348026 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:21.348053 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:50:21.867805 (+519752us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 13:50:21.868013 (+ 208us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.236.254:34489: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":111,"threads_started":1}
I20260926 13:50:21.869187 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16635
2026-09-26T13:50:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (804 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:22 dist-test-slave-r34j krb5kdc[16743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:22 dist-test-slave-r34j krb5kdc[16743](info): set up 2 sockets
Sep 26 13:50:22 dist-test-slave-r34j krb5kdc[16743](info): commencing operation
krb5kdc: starting...
W20260926 13:50:23.204134 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:23 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:23.336226 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:46279
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:23.448966 16756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:23.449267 16756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:23.449318 16756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:23.453361 16756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:23.453418 16756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:23.453437 16756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:23.453454 16756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:23.453469 16756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:23.458796 16756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:46279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:23.459856 16756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:23.460819 16756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:23.469497 16762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:23.469481 16761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:23.469614 16756 server_base.cc:1034] running on GCE node
W20260926 13:50:23.469509 16764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:23.470484 16756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:23.471655 16756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:23.472816 16756 hybrid_clock.cc:648] HybridClock initialized: now 1790430623472788 us; error 47 us; skew 500 ppm
Sep 26 13:50:23 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:23.480011 16756 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:23.481031 16756 webserver.cc:460] Webserver started at http://127.10.236.254:41379/ using document root <none> and password file <none>
I20260926 13:50:23.481621 16756 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:23.481717 16756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:23.481966 16756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:23.484187 16756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "e0f1ddc2864fed30f1c2e216a17af533"
server_key_iv: "593e3f89b8f92f638417a99659fee811"
server_key_version: "encryptionkey@0"
I20260926 13:50:23.484766 16756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "e0f1ddc2864fed30f1c2e216a17af533"
server_key_iv: "593e3f89b8f92f638417a99659fee811"
server_key_version: "encryptionkey@0"
I20260926 13:50:23.488607 16756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:23.491001 16771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:23.492391 16756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:23.492549 16756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "e0f1ddc2864fed30f1c2e216a17af533"
server_key_iv: "593e3f89b8f92f638417a99659fee811"
server_key_version: "encryptionkey@0"
I20260926 13:50:23.492676 16756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:23.518256 16756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:23.521793 16756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:23.522025 16756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:23.529945 16756 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:46279
I20260926 13:50:23.530040 16823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:46279 every 8 connection(s)
I20260926 13:50:23.530970 16756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:23.533490 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16756
I20260926 13:50:23.533629 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:23.533905 11187 external_mini_cluster.cc:1428] Setting key cadbf7e8ac65c71adbe8c83c8b50df19
I20260926 13:50:23.534075 16824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:23.541496 16824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04: Bootstrap starting.
Sep 26 13:50:23 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:23.544059 16824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:23.544948 16824 log.cc:824] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:23.546949 16824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04: No bootstrap required, opened a new log
I20260926 13:50:23.549363 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:23.535351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.10.236.254:46279)
0926 13:50:23.535827 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:23.535846 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:23.535889 (+ 43us) server_negotiation.cc:407] Connection header received
0926 13:50:23.536955 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:23.536980 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:23.537420 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:23.537804 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:23.538519 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:23.540009 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:23.540945 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:23.541626 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:23.541778 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:23.544582 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:23.544588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:23.544605 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:23.544637 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:23.546489 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:23.546956 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:23.546962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:23.546976 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:23.547133 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:23.547424 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:23.547429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:23.547431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:23.547885 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:23.548092 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:23.548369 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 13:50:23.548665 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":182,"threads_started":1}
I20260926 13:50:23.550670 16824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 46279 } }
I20260926 13:50:23.550890 16824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:23.550956 16824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9fd6a4b77c4888a456bce8bf4c0f04, State: Initialized, Role: FOLLOWER
I20260926 13:50:23.551378 16824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 46279 } }
I20260926 13:50:23.551576 16824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:23.551656 16824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:23.551752 16824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:23.552593 16824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 46279 } }
I20260926 13:50:23.552884 16824 leader_election.cc:302] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b9fd6a4b77c4888a456bce8bf4c0f04; no voters:
I20260926 13:50:23.553133 16824 leader_election.cc:288] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:23.553277 16829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:23.553521 16829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [term 1 LEADER]: Becoming Leader. State: Replica: 6b9fd6a4b77c4888a456bce8bf4c0f04, State: Running, Role: LEADER
I20260926 13:50:23.553851 16829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 46279 } }
I20260926 13:50:23.554306 16824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:23.555452 16831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b9fd6a4b77c4888a456bce8bf4c0f04. Latest consensus state: current_term: 1 leader_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 46279 } } }
I20260926 13:50:23.555609 16831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:23.555930 16830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9fd6a4b77c4888a456bce8bf4c0f04" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 46279 } } }
I20260926 13:50:23.556104 16830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:23.556020 16838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:23.559008 16838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:23.564675 16838 catalog_manager.cc:1347] Generated new cluster ID: c6655a04af2a4e1c91b03c46f3591f7a
I20260926 13:50:23.564744 16838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:23.571717 16838 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:23.572932 16838 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:23.584748 16838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b9fd6a4b77c4888a456bce8bf4c0f04: Generated new TSK 0
I20260926 13:50:23.585633 16838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:23.627270 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:46279
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:23.743382 16850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:23.743661 16850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:23.743744 16850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:23.747789 16850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:23.747859 16850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:23.747918 16850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:23.753296 16850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:23.754393 16850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:23.755527 16850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:23.765748 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:23.765780 16855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:23.765759 16858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:23.766028 16850 server_base.cc:1034] running on GCE node
I20260926 13:50:23.766579 16850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:23.767285 16850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:23.768492 16850 hybrid_clock.cc:648] HybridClock initialized: now 1790430623768468 us; error 44 us; skew 500 ppm
Sep 26 13:50:23 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:23.771366 16850 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:23.772277 16850 webserver.cc:460] Webserver started at http://127.10.236.193:34363/ using document root <none> and password file <none>
I20260926 13:50:23.772890 16850 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:23.772989 16850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:23.773242 16850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:23.775297 16850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "e9da6e48414c41649dd3b817ff02143c"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "3cc7d8643360a4e9ca2b27f6af7db4a6"
server_key_iv: "e08c25d14e4b220b89d85491a266c897"
server_key_version: "encryptionkey@0"
I20260926 13:50:23.775938 16850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "e9da6e48414c41649dd3b817ff02143c"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "3cc7d8643360a4e9ca2b27f6af7db4a6"
server_key_iv: "e08c25d14e4b220b89d85491a266c897"
server_key_version: "encryptionkey@0"
I20260926 13:50:23.779953 16850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:50:23.782601 16865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:23.784130 16850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:23.784286 16850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "e9da6e48414c41649dd3b817ff02143c"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "3cc7d8643360a4e9ca2b27f6af7db4a6"
server_key_iv: "e08c25d14e4b220b89d85491a266c897"
server_key_version: "encryptionkey@0"
I20260926 13:50:23.784416 16850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:23.800597 16850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:23.804113 16850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:23.804392 16850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:23.805003 16850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:23.805842 16850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:23.805918 16850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:23.805994 16850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:23.806039 16850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:23.815361 16850 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:34029
I20260926 13:50:23.815379 16978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:34029 every 8 connection(s)
I20260926 13:50:23.816310 16850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:23.824448 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16850
I20260926 13:50:23.824599 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:23.824921 11187 external_mini_cluster.cc:1428] Setting key 16edf24e194a8ec3e0010ddc85579e8c
Sep 26 13:50:23 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:23.830922 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:23.818203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57497 (local address 127.10.236.254:46279)
0926 13:50:23.818362 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:23.818367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:23.819232 (+ 865us) server_negotiation.cc:407] Connection header received
0926 13:50:23.820556 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:23.820559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:23.820605 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:23.820680 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:23.821829 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:23.822384 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:23.823639 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:23.823888 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:23.823967 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:23.826817 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:23.826820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:23.826822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:23.826844 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:23.828570 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:23.829142 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:23.829146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:23.829149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:23.829260 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:23.829641 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:23.829647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:23.829652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:23.829903 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:23.830005 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:23.830527 (+ 522us) server_negotiation.cc:299] Negotiation successful
0926 13:50:23.830675 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 13:50:23.831764 16981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:23.818604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:46279 (local address 127.10.236.193:57497)
0926 13:50:23.819035 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:23.819077 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:23.820369 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:23.820839 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:23.820846 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:23.821238 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:23.821706 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:23.821719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:23.822517 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:23.822520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:23.823445 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:23.823454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:23.824038 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:23.824041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:23.824270 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:23.824922 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:23.824938 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:23.826648 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:23.828711 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:23.828716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:23.828728 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:23.828987 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:23.829370 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:23.829375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:23.829378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:23.829530 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:23.830025 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:23.830030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:23.830268 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 13:50:23.830439 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 13:50:23.830761 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":103,"threads_started":1}
I20260926 13:50:23.833081 16979 heartbeater.cc:347] Connected to a master server at 127.10.236.254:46279
I20260926 13:50:23.833335 16979 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:23.833935 16979 heartbeater.cc:511] Master 127.10.236.254:46279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:50:23.835572 16788 ts_manager.cc:194] Registered new tserver with Master: e9da6e48414c41649dd3b817ff02143c (127.10.236.193:34029)
I20260926 13:50:23.836701 16788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:57497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:23.856396 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:46279
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:23.968124 16984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:23.968405 16984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:23.968448 16984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:23.972891 16984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:23.972963 16984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:23.973026 16984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:23.978577 16984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:23.979794 16984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:23.980785 16984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:23.990617 16990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:23.990608 16989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:23.990746 16984 server_base.cc:1034] running on GCE node
W20260926 13:50:23.990612 16992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:23.991387 16984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:23.992072 16984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:23.993265 16984 hybrid_clock.cc:648] HybridClock initialized: now 1790430623993253 us; error 32 us; skew 500 ppm
Sep 26 13:50:23 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:23.996239 16984 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:23.997254 16984 webserver.cc:460] Webserver started at http://127.10.236.194:41945/ using document root <none> and password file <none>
I20260926 13:50:23.997840 16984 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:23.997936 16984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:23.998172 16984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:24.000310 16984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "00de4bcd955041b4a8e0996c321ed2ef"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "501d7725a3acedb9538c7b0e543cb54b"
server_key_iv: "9afd01d5e05653335067c20f34eca9e6"
server_key_version: "encryptionkey@0"
I20260926 13:50:24.000880 16984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "00de4bcd955041b4a8e0996c321ed2ef"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "501d7725a3acedb9538c7b0e543cb54b"
server_key_iv: "9afd01d5e05653335067c20f34eca9e6"
server_key_version: "encryptionkey@0"
I20260926 13:50:24.004755 16984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:24.007272 16999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:24.008606 16984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:24.008759 16984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "00de4bcd955041b4a8e0996c321ed2ef"
format_stamp: "Formatted at 2026-09-26 13:50:23 on dist-test-slave-r34j"
server_key: "501d7725a3acedb9538c7b0e543cb54b"
server_key_iv: "9afd01d5e05653335067c20f34eca9e6"
server_key_version: "encryptionkey@0"
I20260926 13:50:24.008883 16984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:24.029150 16984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:24.032812 16984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:24.033051 16984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:24.033622 16984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:24.034417 16984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:24.034463 16984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:24.034502 16984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:24.034518 16984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:24.043233 16984 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:34121
I20260926 13:50:24.043254 17112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:34121 every 8 connection(s)
I20260926 13:50:24.044195 16984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:24.053460 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 16984
I20260926 13:50:24.053586 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:24.053877 11187 external_mini_cluster.cc:1428] Setting key 7a375d0f8986c79379a651247e169f61
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:24.059892 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.046135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:52897 (local address 127.10.236.254:46279)
0926 13:50:24.046311 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.046315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.047176 (+ 861us) server_negotiation.cc:407] Connection header received
0926 13:50:24.048643 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.048646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.048693 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.048765 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.050156 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.050712 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.052048 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.052270 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.052361 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.055283 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:24.055288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:24.055294 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:24.055318 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:24.057398 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.058105 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.058108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.058109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.058232 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.058649 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.058652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.058653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.058829 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:24.058907 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:24.059651 (+ 744us) server_negotiation.cc:299] Negotiation successful
0926 13:50:24.059769 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:24.061188 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.046536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:46279 (local address 127.10.236.194:52897)
0926 13:50:24.047009 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:24.047053 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:24.048387 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:24.049009 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:24.049020 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:24.049424 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:24.049946 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:24.049963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.050845 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.050848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:24.051889 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:24.051898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.052417 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.052420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:24.052566 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.053165 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:24.053219 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:24.055097 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:24.057544 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:24.057554 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:24.057582 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:24.057989 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:24.058335 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:24.058340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:24.058344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:24.058547 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:24.058941 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:24.058950 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:24.059345 (+ 395us) client_negotiation.cc:769] Sending connection context
0926 13:50:24.059587 (+ 242us) client_negotiation.cc:240] Negotiation successful
0926 13:50:24.059966 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":112,"threads_started":1}
I20260926 13:50:24.062597 17113 heartbeater.cc:347] Connected to a master server at 127.10.236.254:46279
I20260926 13:50:24.062871 17113 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:24.063374 17113 heartbeater.cc:511] Master 127.10.236.254:46279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:24.064520 16788 ts_manager.cc:194] Registered new tserver with Master: 00de4bcd955041b4a8e0996c321ed2ef (127.10.236.194:34121)
I20260926 13:50:24.065119 16788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:52897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:24.087546 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:46279
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:24.204486 17118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:24.204754 17118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:24.204804 17118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:24.208804 17118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:24.208866 17118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:24.208925 17118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:24.214349 17118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:24.215358 17118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:24.216439 17118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:24.226512 17126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:24.226468 17124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:24.226468 17123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:24.226491 17118 server_base.cc:1034] running on GCE node
I20260926 13:50:24.227155 17118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:24.227875 17118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:24.229049 17118 hybrid_clock.cc:648] HybridClock initialized: now 1790430624229036 us; error 26 us; skew 500 ppm
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.232039 17118 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:24.233116 17118 webserver.cc:460] Webserver started at http://127.10.236.195:37813/ using document root <none> and password file <none>
I20260926 13:50:24.233767 17118 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:24.233835 17118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:24.234081 17118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:24.236214 17118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "b196d63fa3964e6690a3853f0036d484"
format_stamp: "Formatted at 2026-09-26 13:50:24 on dist-test-slave-r34j"
server_key: "e45f74f3cb226e31ceaa17f69d5fab32"
server_key_iv: "3e9447aec5083b1ee1d12bcd2d7f070f"
server_key_version: "encryptionkey@0"
I20260926 13:50:24.236802 17118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "b196d63fa3964e6690a3853f0036d484"
format_stamp: "Formatted at 2026-09-26 13:50:24 on dist-test-slave-r34j"
server_key: "e45f74f3cb226e31ceaa17f69d5fab32"
server_key_iv: "3e9447aec5083b1ee1d12bcd2d7f070f"
server_key_version: "encryptionkey@0"
I20260926 13:50:24.240836 17118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:50:24.243398 17133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:24.245077 17118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:50:24.245223 17118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "b196d63fa3964e6690a3853f0036d484"
format_stamp: "Formatted at 2026-09-26 13:50:24 on dist-test-slave-r34j"
server_key: "e45f74f3cb226e31ceaa17f69d5fab32"
server_key_iv: "3e9447aec5083b1ee1d12bcd2d7f070f"
server_key_version: "encryptionkey@0"
I20260926 13:50:24.245345 17118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:24.260732 17118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:24.263914 17118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:24.264156 17118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:24.264757 17118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:24.265568 17118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:24.265642 17118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:24.265743 17118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:24.265796 17118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:24.275100 17118 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:46131
I20260926 13:50:24.275136 17246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:46131 every 8 connection(s)
I20260926 13:50:24.276127 17118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:24.284782 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17118
I20260926 13:50:24.284912 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:24.285238 11187 external_mini_cluster.cc:1428] Setting key ce755ed9e108441be4803ddcb7758118
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:24.290695 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.278084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:55353 (local address 127.10.236.254:46279)
0926 13:50:24.278234 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.278238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.279037 (+ 799us) server_negotiation.cc:407] Connection header received
0926 13:50:24.280334 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.280338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.280383 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.280471 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.281650 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.282315 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.283641 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.283886 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.283981 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.286743 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:24.286746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:24.286748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:24.286772 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:24.288568 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.289098 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.289103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.289105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.289194 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.289540 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.289545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.289547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.289748 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:24.289857 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:24.290403 (+ 546us) server_negotiation.cc:299] Negotiation successful
0926 13:50:24.290527 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:50:24.291661 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.278457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:46279 (local address 127.10.236.195:55353)
0926 13:50:24.278886 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:24.278927 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:24.280128 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:24.280634 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:24.280641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:24.281004 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:24.281497 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:24.281511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.282457 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.282459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:24.283419 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:24.283429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.284044 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.284047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:24.284232 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.284817 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:24.284841 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:24.286541 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:24.288699 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:24.288705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:24.288721 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:24.288988 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:24.289290 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:24.289294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:24.289296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:24.289443 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:24.289867 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:24.289876 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:24.290147 (+ 271us) client_negotiation.cc:769] Sending connection context
0926 13:50:24.290356 (+ 209us) client_negotiation.cc:240] Negotiation successful
0926 13:50:24.290689 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":100,"threads_started":1}
I20260926 13:50:24.292869 17247 heartbeater.cc:347] Connected to a master server at 127.10.236.254:46279
I20260926 13:50:24.293110 17247 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:24.293596 17247 heartbeater.cc:511] Master 127.10.236.254:46279 requested a full tablet report, sending...
I20260926 13:50:24.294634 16788 ts_manager.cc:194] Registered new tserver with Master: b196d63fa3964e6690a3853f0036d484 (127.10.236.195:46131)
I20260926 13:50:24.295272 16788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:55353
I20260926 13:50:24.299700 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/test-admin.keytab.
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.317066 11187 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 13:50:24.319942 17254 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:24.318542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.10.236.193:34029)
0926 13:50:24.318823 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.318836 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.318858 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:50:24.318906 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.318911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.319095 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.319213 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.319734 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59790: BlockingRecv error: recv got EOF from 127.0.0.1:59790 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
W20260926 13:50:24.320226 11187 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc)
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.321273 11187 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.322235 11187 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:50:24.332396 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.323148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.10.236.193:34029)
0926 13:50:24.323325 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.323329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.323445 (+ 116us) server_negotiation.cc:407] Connection header received
0926 13:50:24.323641 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.323643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.323684 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.323779 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.324328 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.325086 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.325934 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.326305 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.326424 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.328778 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:24.328788 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:24.328805 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:24.328827 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:24.330467 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.330894 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.330901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.330903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.331021 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.331278 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.331289 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.331291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.331617 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:24.331831 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:24.332052 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 13:50:24.332199 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 13:50:24.336211 17254 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:24.335076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.10.236.193:34029)
0926 13:50:24.335206 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.335210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.335352 (+ 142us) server_negotiation.cc:407] Connection header received
0926 13:50:24.335527 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.335530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.335570 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.335658 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.336090 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59812: BlockingRecv error: recv got EOF from 127.0.0.1:59812 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 13:50:24.336377 11187 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc' of type FILE
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.337404 11187 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.338289 11187 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:50:24.346796 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.339104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.10.236.193:34029)
0926 13:50:24.339249 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.339252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.339335 (+ 83us) server_negotiation.cc:407] Connection header received
0926 13:50:24.339425 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.339427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.339460 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.339578 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.340034 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.340548 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.341314 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.341525 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.341597 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.343722 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:24.343725 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:24.343727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:24.343747 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:24.345203 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.345597 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.345599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.345601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.345667 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.346001 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.346003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.346004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.346151 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:24.346245 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:24.346492 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 13:50:24.346582 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 13:50:24.349964 17254 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:24.348948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.10.236.193:34029)
0926 13:50:24.349050 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.349072 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.349216 (+ 144us) server_negotiation.cc:407] Connection header received
0926 13:50:24.349375 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.349377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.349412 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.349476 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.349851 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59824: BlockingRecv error: recv got EOF from 127.0.0.1:59824 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260926 13:50:24.350242 11187 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790430594225530-11187-0/krb5kdc/krb5cc' of type FILE
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.351280 11187 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:24.352182 11187 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:50:24.360755 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:24.352951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.10.236.193:34029)
0926 13:50:24.353089 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:24.353093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:24.353210 (+ 117us) server_negotiation.cc:407] Connection header received
0926 13:50:24.353351 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:24.353354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:24.353392 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:24.353484 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:24.353965 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.354543 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.355411 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:24.355700 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:24.355807 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:24.357936 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:24.357940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:24.357943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:24.357972 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:24.359166 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.359607 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.359612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.359614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.359692 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:24.359968 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:24.359973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:24.359975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:24.360132 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:24.360274 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:24.360501 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 13:50:24.360598 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 13:50:24.362028 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16850
I20260926 13:50:24.368049 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16984
I20260926 13:50:24.373207 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17118
I20260926 13:50:24.378407 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 16756
2026-09-26T13:50:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2254 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[17267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[17267](info): set up 2 sockets
Sep 26 13:50:24 dist-test-slave-r34j krb5kdc[17267](info): commencing operation
krb5kdc: starting...
W20260926 13:50:25.428574 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:25 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab.
Entry for principal oryx/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab.
I20260926 13:50:25.562440 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:36389
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab
--principal=oryx/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:25.672677 17280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:25.673024 17280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:25.673111 17280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:25.677285 17280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:25.677389 17280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:25.677428 17280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:25.677462 17280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:25.677500 17280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:25.682937 17280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:36389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab
--principal=oryx/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:25.684196 17280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:25.685252 17280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:25.694093 17285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:25.694161 17286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:25.694103 17288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:25.694247 17280 server_base.cc:1034] running on GCE node
I20260926 13:50:25.695024 17280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:25.696277 17280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:25.697491 17280 hybrid_clock.cc:648] HybridClock initialized: now 1790430625697473 us; error 34 us; skew 500 ppm
Sep 26 13:50:25 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:25.700973 17280 init.cc:377] Logged in from keytab as oryx/127.10.236.254@KRBTEST.COM (short username oryx)
I20260926 13:50:25.701964 17280 webserver.cc:460] Webserver started at http://127.10.236.254:46031/ using document root <none> and password file <none>
I20260926 13:50:25.702594 17280 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:25.702659 17280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:25.702833 17280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:25.705052 17280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "4f95ea2a4aba495e9221893b3cc9aa92"
format_stamp: "Formatted at 2026-09-26 13:50:25 on dist-test-slave-r34j"
server_key: "bd3a0e05b25ddd57e7cffc0cda26e9f5"
server_key_iv: "291d406c03c27fd10d25591415e130e2"
server_key_version: "encryptionkey@0"
I20260926 13:50:25.705600 17280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "4f95ea2a4aba495e9221893b3cc9aa92"
format_stamp: "Formatted at 2026-09-26 13:50:25 on dist-test-slave-r34j"
server_key: "bd3a0e05b25ddd57e7cffc0cda26e9f5"
server_key_iv: "291d406c03c27fd10d25591415e130e2"
server_key_version: "encryptionkey@0"
I20260926 13:50:25.709728 17280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:25.712766 17295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:25.714370 17280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:50:25.714558 17280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "4f95ea2a4aba495e9221893b3cc9aa92"
format_stamp: "Formatted at 2026-09-26 13:50:25 on dist-test-slave-r34j"
server_key: "bd3a0e05b25ddd57e7cffc0cda26e9f5"
server_key_iv: "291d406c03c27fd10d25591415e130e2"
server_key_version: "encryptionkey@0"
I20260926 13:50:25.714710 17280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:25.743345 17280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:25.746908 17280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:25.747155 17280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:25.754482 17280 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:36389
I20260926 13:50:25.754788 17347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:36389 every 8 connection(s)
I20260926 13:50:25.755652 17280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:25.758551 17348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:25.759627 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17280
I20260926 13:50:25.759795 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:25.760092 11187 external_mini_cluster.cc:1428] Setting key 9710242f9877f77dcde5d626f00cc3df
I20260926 13:50:25.766148 17348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Bootstrap starting.
I20260926 13:50:25.768565 17348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Neither blocks nor log segments found. Creating new log.
Sep 26 13:50:25 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:25.769407 17348 log.cc:824] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:25.771396 17348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: No bootstrap required, opened a new log
I20260926 13:50:25.774673 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:25.761470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.10.236.254:36389)
0926 13:50:25.761905 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:25.761921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:25.761960 (+ 39us) server_negotiation.cc:407] Connection header received
0926 13:50:25.762684 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:25.762703 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:25.763070 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:25.763437 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:25.764332 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:25.765332 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:25.766218 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:25.766765 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:25.766956 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:25.769621 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:25.769627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:25.769642 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:25.769672 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:25.771447 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:25.772041 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:25.772046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:25.772051 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:25.772164 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:25.772499 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:25.772502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:25.772503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:25.772980 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:25.773146 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:25.773492 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 13:50:25.773884 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":111,"threads_started":1}
I20260926 13:50:25.775638 17348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } }
I20260926 13:50:25.775889 17348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:25.775970 17348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f95ea2a4aba495e9221893b3cc9aa92, State: Initialized, Role: FOLLOWER
I20260926 13:50:25.776422 17348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } }
I20260926 13:50:25.776577 17348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:25.776665 17348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:25.776788 17348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:25.777767 17348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } }
I20260926 13:50:25.778122 17348 leader_election.cc:302] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f95ea2a4aba495e9221893b3cc9aa92; no voters:
I20260926 13:50:25.778525 17348 leader_election.cc:288] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:25.778601 17353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:25.778843 17353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 LEADER]: Becoming Leader. State: Replica: 4f95ea2a4aba495e9221893b3cc9aa92, State: Running, Role: LEADER
I20260926 13:50:25.779196 17353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } }
I20260926 13:50:25.780071 17348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:25.781136 17354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f95ea2a4aba495e9221893b3cc9aa92. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } }
I20260926 13:50:25.781178 17355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } }
I20260926 13:50:25.781275 17354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:25.781297 17355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:25.782549 17362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:25.785300 17362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:25.792088 17362 catalog_manager.cc:1347] Generated new cluster ID: 614021588dc141c18498e5ceaef77458
I20260926 13:50:25.792189 17362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:25.809716 17362 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:25.810753 17362 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:25.819415 17362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Generated new TSK 0
I20260926 13:50:25.820144 17362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab.
Entry for principal oryx/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab.
I20260926 13:50:25.855562 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36389
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:25.972963 17374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:25.973255 17374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:25.973300 17374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:25.977996 17374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:25.978070 17374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:25.978170 17374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:25.984166 17374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:25.985321 17374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:25.986392 17374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:25.996142 17380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:25.996156 17382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:25.996137 17379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:25.996404 17374 server_base.cc:1034] running on GCE node
I20260926 13:50:25.996869 17374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:25.997572 17374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:25.998762 17374 hybrid_clock.cc:648] HybridClock initialized: now 1790430625998734 us; error 40 us; skew 500 ppm
Sep 26 13:50:25 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:26.001915 17374 init.cc:377] Logged in from keytab as oryx/127.10.236.193@KRBTEST.COM (short username oryx)
I20260926 13:50:26.002877 17374 webserver.cc:460] Webserver started at http://127.10.236.193:40295/ using document root <none> and password file <none>
I20260926 13:50:26.003545 17374 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:26.003612 17374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:26.003827 17374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:26.005911 17374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "9c08d8b4d0874f5e948507bb37f8ff38"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "f5b3b6a924633c508975cc88fa6edd7e"
server_key_iv: "28b8c6d20778fb0940fc288c5d3168ba"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.006487 17374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "9c08d8b4d0874f5e948507bb37f8ff38"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "f5b3b6a924633c508975cc88fa6edd7e"
server_key_iv: "28b8c6d20778fb0940fc288c5d3168ba"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.010344 17374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260926 13:50:26.012969 17389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.014331 17374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:26.014469 17374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "9c08d8b4d0874f5e948507bb37f8ff38"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "f5b3b6a924633c508975cc88fa6edd7e"
server_key_iv: "28b8c6d20778fb0940fc288c5d3168ba"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.014587 17374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:26.031040 17374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:26.034222 17374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:26.034435 17374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:26.035104 17374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:26.035971 17374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:26.036049 17374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.036116 17374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:26.036172 17374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.045122 17374 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:36001
I20260926 13:50:26.045142 17502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:36001 every 8 connection(s)
I20260926 13:50:26.046027 17374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:26.052850 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17374
I20260926 13:50:26.052975 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:26.053299 11187 external_mini_cluster.cc:1428] Setting key df999c830e49167aa35fe6a2d044f754
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:26.061563 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.047961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:35115 (local address 127.10.236.254:36389)
0926 13:50:26.048102 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.048105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.049013 (+ 908us) server_negotiation.cc:407] Connection header received
0926 13:50:26.050410 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.050414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.050473 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.050583 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.051868 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.052414 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.054015 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.054256 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.054389 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.057268 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.057274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.057280 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.057316 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.059218 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.059829 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.059834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.059836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.059936 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.060285 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.060291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.060294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.060578 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.060690 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.061200 (+ 510us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.061363 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 13:50:26.062392 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.048368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:35115)
0926 13:50:26.048847 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.048889 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.050179 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.050763 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.050770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.051128 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.051708 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.051724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.052556 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.052559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.053888 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.053898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.054434 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.054438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.054620 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.055226 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.055258 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.057084 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.059370 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.059378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.059397 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.059723 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.060031 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.060033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.060034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.060171 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.060690 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.060695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.060934 (+ 239us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.061102 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.061399 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for oryx/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:50:26.063884 17503 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36389
I20260926 13:50:26.064201 17503 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:26.064708 17503 heartbeater.cc:511] Master 127.10.236.254:36389 requested a full tablet report, sending...
I20260926 13:50:26.066226 17312 ts_manager.cc:194] Registered new tserver with Master: 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.194@KRBTEST.COM" created.
I20260926 13:50:26.067380 17312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:35115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab.
Entry for principal oryx/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab.
I20260926 13:50:26.083655 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36389
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:26.195163 17508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:26.195464 17508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:26.195632 17508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:26.199630 17508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:26.199728 17508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:26.199826 17508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:26.205083 17508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:26.206216 17508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:26.207278 17508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:26.217185 17514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:26.217108 17513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.217298 17508 server_base.cc:1034] running on GCE node
W20260926 13:50:26.217084 17516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.217886 17508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:26.218715 17508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:26.219920 17508 hybrid_clock.cc:648] HybridClock initialized: now 1790430626219885 us; error 44 us; skew 500 ppm
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:26.222872 17508 init.cc:377] Logged in from keytab as oryx/127.10.236.194@KRBTEST.COM (short username oryx)
I20260926 13:50:26.224018 17508 webserver.cc:460] Webserver started at http://127.10.236.194:44897/ using document root <none> and password file <none>
I20260926 13:50:26.224713 17508 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:26.224819 17508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:26.225076 17508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:26.227311 17508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "396ab5e82406cf7a5877fea707fc7d9f"
server_key_iv: "5a62a38bbfe180f69f1e880e7880b8da"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.228039 17508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "396ab5e82406cf7a5877fea707fc7d9f"
server_key_iv: "5a62a38bbfe180f69f1e880e7880b8da"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.232122 17508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:26.234601 17523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.236080 17508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:26.236174 17508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "396ab5e82406cf7a5877fea707fc7d9f"
server_key_iv: "5a62a38bbfe180f69f1e880e7880b8da"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.236260 17508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:26.249598 17508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:26.252840 17508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:26.253067 17508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:26.253681 17508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:26.254525 17508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:26.254572 17508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.254612 17508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:26.254628 17508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.264019 17508 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:36329
I20260926 13:50:26.264087 17636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:36329 every 8 connection(s)
I20260926 13:50:26.264962 17508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:26.270241 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17508
I20260926 13:50:26.270433 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:26.270736 11187 external_mini_cluster.cc:1428] Setting key 13409fc20e2ce550725dd48d2dd657b5
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:26.280028 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.266930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:45681 (local address 127.10.236.254:36389)
0926 13:50:26.267071 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.267074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.267928 (+ 854us) server_negotiation.cc:407] Connection header received
0926 13:50:26.269371 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.269374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.269435 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.269542 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.270935 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.271563 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.272704 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.272928 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.273039 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.275985 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.275991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.275997 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.276032 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.277748 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.278250 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.278255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.278260 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.278364 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.278713 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.278719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.278722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.279002 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.279118 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.279720 (+ 602us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.279860 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:50:26.281068 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.267287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:45681)
0926 13:50:26.267776 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.267818 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.269062 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.269714 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.269722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.270099 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.270751 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.270766 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.271688 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.271692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.272572 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.272581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.273090 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.273094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.273269 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.273953 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.273980 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.275806 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.277868 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.277873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.277886 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.278138 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.278470 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.278472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.278473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.278611 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.279115 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.279120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.279370 (+ 250us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.279680 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.280012 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for oryx/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:26.282323 17637 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36389
I20260926 13:50:26.282579 17637 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:26.283107 17637 heartbeater.cc:511] Master 127.10.236.254:36389 requested a full tablet report, sending...
I20260926 13:50:26.284206 17312 ts_manager.cc:194] Registered new tserver with Master: 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.195@KRBTEST.COM" created.
I20260926 13:50:26.284906 17312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:45681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab.
Entry for principal oryx/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab.
I20260926 13:50:26.304724 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36389
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:26.421260 17642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:26.421591 17642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:26.421672 17642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:26.426504 17642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:26.426609 17642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:26.426704 17642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:26.432163 17642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:26.433279 17642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:26.434312 17642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:26.444594 17650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:26.444710 17648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:26.444582 17647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.444873 17642 server_base.cc:1034] running on GCE node
I20260926 13:50:26.445463 17642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:26.446182 17642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:26.447423 17642 hybrid_clock.cc:648] HybridClock initialized: now 1790430626447392 us; error 40 us; skew 500 ppm
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:26.454304 17642 init.cc:377] Logged in from keytab as oryx/127.10.236.195@KRBTEST.COM (short username oryx)
I20260926 13:50:26.455300 17642 webserver.cc:460] Webserver started at http://127.10.236.195:35695/ using document root <none> and password file <none>
I20260926 13:50:26.455986 17642 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:26.456086 17642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:26.456332 17642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:26.458408 17642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "9f429a955862402981ded49a581ba755"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "2d93f812fdc7dd544dfaadc47bc2d8e2"
server_key_iv: "4a55e6907e5b34e8f3804f16c03c930a"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.459009 17642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "9f429a955862402981ded49a581ba755"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "2d93f812fdc7dd544dfaadc47bc2d8e2"
server_key_iv: "4a55e6907e5b34e8f3804f16c03c930a"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.462993 17642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:26.465824 17657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.467224 17642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:26.467434 17642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "9f429a955862402981ded49a581ba755"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "2d93f812fdc7dd544dfaadc47bc2d8e2"
server_key_iv: "4a55e6907e5b34e8f3804f16c03c930a"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.467633 17642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:26.478026 17642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:26.481865 17642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:26.482086 17642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:26.482764 17642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:26.483681 17642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:26.483757 17642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.483824 17642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:26.483888 17642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.493145 17642 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:42945
I20260926 13:50:26.493167 17770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:42945 every 8 connection(s)
I20260926 13:50:26.494158 17642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:26.502156 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17642
I20260926 13:50:26.502324 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:26.502597 11187 external_mini_cluster.cc:1428] Setting key 07b9d238d7edf77e67d087ee51e8f2c8
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:26.509797 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.496148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45113 (local address 127.10.236.254:36389)
0926 13:50:26.496350 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.496354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.497147 (+ 793us) server_negotiation.cc:407] Connection header received
0926 13:50:26.498534 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.498537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.498595 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.498713 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.500057 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.500920 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.502265 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.502581 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.502705 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.505758 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.505763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.505766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.505796 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.507364 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.508042 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.508045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.508046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.508127 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.508652 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.508656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.508657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.508867 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.508950 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.509569 (+ 619us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.509674 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260926 13:50:26.510820 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.496523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:45113)
0926 13:50:26.496988 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.497030 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.498316 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.498911 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.498919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.499282 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.499874 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.499914 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.501062 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.501066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.502095 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.502107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.502770 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.502775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.503036 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.503837 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.503880 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.505574 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.507575 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.507581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.507595 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.507900 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.508354 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.508357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.508358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.508516 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.509023 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.509029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.509306 (+ 277us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.509477 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.509769 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":114,"threads_started":1}
I20260926 13:50:26.512213 17771 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36389
I20260926 13:50:26.512504 17771 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:26.513057 17771 heartbeater.cc:511] Master 127.10.236.254:36389 requested a full tablet report, sending...
I20260926 13:50:26.514164 17312 ts_manager.cc:194] Registered new tserver with Master: 9f429a955862402981ded49a581ba755 (127.10.236.195:42945)
I20260926 13:50:26.514744 17312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:45113
I20260926 13:50:26.517102 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.236.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.253.keytab.
Entry for principal oryx/127.10.236.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.253.keytab.
I20260926 13:50:26.547724 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:39781
--webserver_interface=127.10.236.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.253.keytab
--principal=oryx/127.10.236.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:36389,127.10.236.253:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:26.673454 17776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:26.673738 17776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:26.673780 17776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:26.677997 17776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:26.678066 17776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:26.678087 17776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:26.678104 17776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:26.678120 17776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:26.683920 17776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:36389,127.10.236.253:39781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.253.keytab
--principal=oryx/127.10.236.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.236.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
W20260926 13:50:26.684165 17776 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.236.254:36389,127.10.236.253:39781'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 13:50:26.685070 17776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:26.686122 17776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:26.695035 17784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.695159 17776 server_base.cc:1034] running on GCE node
W20260926 13:50:26.695072 17781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:26.695035 17782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.696059 17776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:26.697206 17776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:26.698426 17776 hybrid_clock.cc:648] HybridClock initialized: now 1790430626698413 us; error 29 us; skew 500 ppm
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:26.701484 17776 init.cc:377] Logged in from keytab as oryx/127.10.236.253@KRBTEST.COM (short username oryx)
I20260926 13:50:26.702361 17776 webserver.cc:460] Webserver started at http://127.10.236.253:36751/ using document root <none> and password file <none>
I20260926 13:50:26.702936 17776 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:26.702997 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:26.703246 17776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:26.705439 17776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/instance:
uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "e135067095925e4b31b982a5aba5534e"
server_key_iv: "f1eddadc24469700a3299e48dd040947"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.706027 17776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal/instance:
uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "e135067095925e4b31b982a5aba5534e"
server_key_iv: "f1eddadc24469700a3299e48dd040947"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.709949 17776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:26.712560 17791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.713912 17776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:26.714068 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "e135067095925e4b31b982a5aba5534e"
server_key_iv: "f1eddadc24469700a3299e48dd040947"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.714192 17776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:26.726263 17776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:26.729550 17776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:26.729763 17776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:26.737143 17776 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.253:39781
I20260926 13:50:26.737464 17843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.253:39781 every 8 connection(s)
I20260926 13:50:26.738158 17776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
I20260926 13:50:26.741134 17844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:26.745297 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17776
I20260926 13:50:26.745088 17844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.253@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
I20260926 13:50:26.760557 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.746939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.10.236.254:36389)
0926 13:50:26.747074 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.747079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.747700 (+ 621us) server_negotiation.cc:407] Connection header received
0926 13:50:26.748463 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.748466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.748516 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.748597 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.749758 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.750693 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.752310 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.752637 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.752757 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.756144 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.756148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.756151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.756192 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.758099 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.758729 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.758733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.758736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.758818 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.759209 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.759212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.759214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.759514 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.759612 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.760248 (+ 636us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.760406 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:26.761763 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.746931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:42740)
0926 13:50:26.747552 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.747599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.748243 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.748737 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.748744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.749208 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.749572 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.749594 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.750847 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.750850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.752123 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.752136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.752810 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.752813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.752960 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.753806 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.753833 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.755900 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.758229 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.758238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.758250 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.758605 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.758915 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.758920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.758922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.759094 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.759606 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.759612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.759995 (+ 383us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.760985 (+ 990us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.761158 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":39,"spinlock_wait_cycles":62080,"thread_start_us":144,"threads_started":1}
I20260926 13:50:26.761763 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.746931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.10.236.253:39781)
0926 13:50:26.747399 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.747419 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.747463 (+ 44us) server_negotiation.cc:407] Connection header received
0926 13:50:26.748213 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.748230 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.748464 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.748645 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.749706 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.750973 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.752184 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.752811 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.752948 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.756577 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.756586 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.756603 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.756634 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.758546 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.759042 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.759048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.759050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.759120 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.759423 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.759426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.759428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.759995 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.760207 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.760438 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.760743 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":319,"spinlock_wait_cycles":2560,"thread_start_us":144,"threads_started":1}
I20260926 13:50:26.763306 17844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.253@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
I20260926 13:50:26.774581 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.763953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.10.236.253:39781)
0926 13:50:26.766271 (+ 2318us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.766276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.766291 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:26.766327 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.766330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.766373 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.766455 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.766885 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.767675 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.768462 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.768777 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.768869 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.771211 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.771214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.771217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.771246 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.773019 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.773446 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.773448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.773449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.773522 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.773804 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.773806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.773807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.773982 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.774059 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.774282 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.774423 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":2240}
I20260926 13:50:26.774608 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.763897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:49938)
0926 13:50:26.763992 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.764006 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.764081 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.766501 (+ 2420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.766504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.766659 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.766760 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.766765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.767823 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.767826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.768344 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.768352 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.768900 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.768903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.768978 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.769563 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.769580 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.771067 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.773141 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.773145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.773147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.773352 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.773623 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.773626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.773627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.773716 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.774072 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.774075 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.774155 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.774251 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.774369 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 13:50:26.778204 17844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Bootstrap starting.
I20260926 13:50:26.780719 17844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:26.781579 17844 log.cc:824] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:26.783703 17844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: No bootstrap required, opened a new log
I20260926 13:50:26.786738 17844 raft_consensus.cc:348] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } }
I20260926 13:50:26.786981 17844 raft_consensus.cc:374] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:26.787067 17844 raft_consensus.cc:729] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fccd9ad787424bca8fa7f3645c0da8a6, State: Initialized, Role: FOLLOWER
I20260926 13:50:26.787627 17844 consensus_queue.cc:260] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } }
I20260926 13:50:26.788127 17850 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } } }
I20260926 13:50:26.788244 17850 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:26.788867 17844 sys_catalog.cc:564] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 13:50:26.793843 17861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:50:26.793929 17861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:50:26.796255 17776 master_runner.cc:428] Detected that this master fccd9ad787424bca8fa7f3645c0da8a6 is joining an existing cluster
I20260926 13:50:26.796332 17776 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.236.253:39781
I20260926 13:50:26.808075 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.798746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.10.236.254:36389)
0926 13:50:26.798893 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.798898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.799050 (+ 152us) server_negotiation.cc:407] Connection header received
0926 13:50:26.799154 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.799157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.799209 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.799296 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.800011 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.800847 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.802121 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.802473 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.802550 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.804097 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.804102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.804108 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.804138 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.806121 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.806636 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.806640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.806643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.806744 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.807099 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.807103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.807105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.807336 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.807434 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.807798 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.807917 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:26.808071 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.798657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:42756)
0926 13:50:26.798942 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.798956 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.799062 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.799433 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.799436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.799706 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.799867 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.799875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.800993 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.800996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.801986 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.801996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.802700 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.802704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.802790 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.803263 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.803278 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.803909 (+ 631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.806221 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.806228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.806231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.806527 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.806859 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.806862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.806864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.806973 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.807502 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.807505 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.807609 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.807715 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.807855 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":30,"spinlock_wait_cycles":9088,"thread_start_us":233,"threads_started":1}
I20260926 13:50:26.808075 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.799271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.10.236.253:39781)
0926 13:50:26.799387 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.799391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.799697 (+ 306us) server_negotiation.cc:407] Connection header received
0926 13:50:26.799814 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.799816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.799856 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.799947 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.800380 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.800862 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.801684 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.801967 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.802079 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.803993 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.803997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.803999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.804020 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.806040 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.806674 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.806677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.806679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.806756 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.807050 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.807056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.807062 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.807302 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.807519 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.807808 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.807916 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":43}
I20260926 13:50:26.808638 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.799277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:49952)
0926 13:50:26.799615 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.799626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.799736 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.799957 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.799959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.800153 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.800235 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.800287 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.800976 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.800979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.801570 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.801577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.802088 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.802091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.802234 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.803313 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.803325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.803846 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.806175 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.806179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.806181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.806569 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.806849 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.806852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.806854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.806948 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.807508 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.807511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.807623 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.808094 (+ 471us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.808425 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":42,"thread_start_us":79,"threads_started":1}
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.254@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
I20260926 13:50:26.824254 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.813627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.10.236.253:39781)
0926 13:50:26.813748 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.813762 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.814565 (+ 803us) server_negotiation.cc:407] Connection header received
0926 13:50:26.814840 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.814846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.814902 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.815012 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.815891 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.816672 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.817864 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.818154 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.818495 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.820791 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.820795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.820797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.820821 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.822304 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.822809 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.822814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.822816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.822895 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.823207 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.823210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.823212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.823393 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.823539 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.823961 (+ 422us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.824079 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:26.824259 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.813552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:49958)
0926 13:50:26.814439 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.814494 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.814664 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.815062 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.815068 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.815616 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.815758 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.815768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.816804 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.816810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.817689 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.817699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.818296 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.818300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.818411 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.818996 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.819008 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.820638 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.822437 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.822446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.822453 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.822700 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.822990 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.822994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.822996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.823086 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.823562 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.823570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.823730 (+ 160us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.823910 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.824107 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for oryx/127.10.236.252@KRBTEST.COM; defaulting to no policy
I20260926 13:50:26.826066 17312 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.236.253:39781: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
member_type: NON_VOTER
last_known_addr {
host: "127.10.236.253"
port: 39781
}
attrs {
promote: true
}
}
dest_uuid: "4f95ea2a4aba495e9221893b3cc9aa92"
cas_config_opid_index: -1
I20260926 13:50:26.826822 17312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.252@KRBTEST.COM" created.
I20260926 13:50:26.829051 17872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } }
I20260926 13:50:26.829761 17818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:26.831354 17818 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Refusing update from remote peer 4f95ea2a4aba495e9221893b3cc9aa92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 13:50:26.831699 17872 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.236.253:39781
I20260926 13:50:26.832178 17776 master.cc:537] Master@127.10.236.253:39781 shutting down...
I20260926 13:50:26.832597 17875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 13:50:26.834357 17299 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 13:50:26.835317 17299 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 -> Peer fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253:39781): Couldn't send request to peer fccd9ad787424bca8fa7f3645c0da8a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 13:50:26.836134 17873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } }
I20260926 13:50:26.836280 17873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:26.837863 17776 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:26.838158 17776 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:26.838284 17776 tablet_replica.cc:330] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: stopping tablet replica
I20260926 13:50:26.842777 17776 master.cc:552] Master@127.10.236.253:39781 shutdown complete.
I20260926 13:50:26.842851 17776 master_runner.cc:305] Clearing existing system tablet
I20260926 13:50:26.843086 17776 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:26.843160 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:26.844377 17776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:50:26.845449 17879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.845651 17776 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:26.845808 17776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:26.845852 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "e135067095925e4b31b982a5aba5534e"
server_key_iv: "f1eddadc24469700a3299e48dd040947"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.252.keytab.
Entry for principal oryx/127.10.236.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.252.keytab.
I20260926 13:50:26.848413 17776 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:26.849020 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:34915
--webserver_interface=127.10.236.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.252.keytab
--principal=oryx/127.10.236.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:50:26.850970 17776 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 13:50:26.851071 17776 log.cc:1196] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 13:50:26.851599 17776 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Deleting consensus metadata
I20260926 13:50:26.851801 17776 master_runner.cc:315] Copying system tablet from 127.10.236.254:36389
I20260926 13:50:26.852962 17776 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: tablet copy: Beginning tablet copy session from remote peer at address 127.10.236.254:36389
I20260926 13:50:26.861152 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.853612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.10.236.254:36389)
0926 13:50:26.853752 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:26.853758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:26.854056 (+ 298us) server_negotiation.cc:407] Connection header received
0926 13:50:26.854205 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:26.854208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:26.854265 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:26.854356 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:26.854809 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.855345 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.856266 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.856505 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.856617 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.857886 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:26.857890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:26.857893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:26.857925 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:26.859423 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.859915 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.859921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.859923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.860022 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:26.860309 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:26.860314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:26.860316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:26.860496 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:26.860607 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:26.860871 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 13:50:26.860983 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:50:26.861161 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:26.853727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:42762)
0926 13:50:26.853941 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:26.853957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:26.854104 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:26.854376 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:26.854379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:26.854581 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:26.854672 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.854678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.855457 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.855462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.856141 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:26.856150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:26.856623 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:26.856626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:26.856718 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:26.857405 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:26.857417 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:26.857725 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:26.859588 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.859591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.859594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.859809 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.860120 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:26.860125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:26.860127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:26.860216 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:26.860595 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:26.860598 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:26.860702 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 13:50:26.860813 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 13:50:26.860958 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260926 13:50:26.861832 17332 tablet_copy_service.cc:140] P 4f95ea2a4aba495e9221893b3cc9aa92: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fccd9ad787424bca8fa7f3645c0da8a6 ({username='oryx', principal='oryx/127.10.236.253@KRBTEST.COM'} at 127.0.0.1:42762)
I20260926 13:50:26.862020 17332 tablet_copy_service.cc:161] P 4f95ea2a4aba495e9221893b3cc9aa92: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fccd9ad787424bca8fa7f3645c0da8a6 at {username='oryx', principal='oryx/127.10.236.253@KRBTEST.COM'} at 127.0.0.1:42762: session id = fccd9ad787424bca8fa7f3645c0da8a6-00000000000000000000000000000000
I20260926 13:50:26.863411 17332 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 13:50:26.864995 17776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 13:50:26.866848 17776 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: tablet copy: Starting download of 0 data blocks...
I20260926 13:50:26.867070 17776 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: tablet copy: Starting download of 1 WAL segments...
I20260926 13:50:26.868814 17776 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 13:50:26.870477 17332 tablet_copy_service.cc:342] P 4f95ea2a4aba495e9221893b3cc9aa92: Request end of tablet copy session fccd9ad787424bca8fa7f3645c0da8a6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.236.253@KRBTEST.COM'} at 127.0.0.1:42762
I20260926 13:50:26.870606 17332 tablet_copy_service.cc:433] P 4f95ea2a4aba495e9221893b3cc9aa92: ending tablet copy session fccd9ad787424bca8fa7f3645c0da8a6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fccd9ad787424bca8fa7f3645c0da8a6
I20260926 13:50:26.877440 17776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:26.879354 17887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:26.879632 17888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.879876 17776 server_base.cc:1034] running on GCE node
W20260926 13:50:26.879981 17890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:26.880313 17776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:26.880812 17776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:26.882066 17776 hybrid_clock.cc:648] HybridClock initialized: now 1790430626882042 us; error 42 us; skew 500 ppm
Sep 26 13:50:26 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430626, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:26.883687 17776 init.cc:377] Logged in from keytab as oryx/127.10.236.253@KRBTEST.COM (short username oryx)
I20260926 13:50:26.884231 17776 webserver.cc:460] Webserver started at http://127.10.236.253:43655/ using document root <none> and password file <none>
I20260926 13:50:26.884435 17776 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:26.884495 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:26.885645 17776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:26.886605 17897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:26.886843 17776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:26.886919 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "e135067095925e4b31b982a5aba5534e"
server_key_iv: "f1eddadc24469700a3299e48dd040947"
server_key_version: "encryptionkey@0"
I20260926 13:50:26.887002 17776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:26.909834 17776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:26.912035 17776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:26.917021 17776 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.253:39781
I20260926 13:50:26.917279 17949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.253:39781 every 8 connection(s)
I20260926 13:50:26.917738 17776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
I20260926 13:50:26.919621 17950 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:50:26.920514 17950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Bootstrap starting.
I20260926 13:50:26.928891 17950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:50:26.929243 17950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: Bootstrap complete.
I20260926 13:50:26.929755 17950 raft_consensus.cc:348] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } }
I20260926 13:50:26.929896 17950 raft_consensus.cc:729] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fccd9ad787424bca8fa7f3645c0da8a6, State: Initialized, Role: LEARNER
I20260926 13:50:26.930006 17950 consensus_queue.cc:260] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } }
I20260926 13:50:26.930362 17953 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } }
I20260926 13:50:26.930549 17950 sys_catalog.cc:564] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:26.930567 17953 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:50:26.933422 17964 catalog_manager.cc:1259] Loaded cluster ID: 614021588dc141c18498e5ceaef77458
I20260926 13:50:26.933743 17964 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: loading cluster ID for follower catalog manager: success
I20260926 13:50:26.935423 17964 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: acquiring CA information for follower catalog manager: success
I20260926 13:50:26.936789 17964 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 13:50:27.006489 17880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:27.006760 17880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:27.006804 17880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:27.011067 17880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:27.011140 17880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:27.011162 17880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:27.011179 17880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:27.011194 17880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:27.016927 17880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.252.keytab
--principal=oryx/127.10.236.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:34915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.236.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:27.018096 17880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:27.019219 17880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:27.028146 17880 server_base.cc:1034] running on GCE node
W20260926 13:50:27.028100 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.028100 17972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.028102 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:27.029121 17880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:27.030257 17880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:27.031461 17880 hybrid_clock.cc:648] HybridClock initialized: now 1790430627031435 us; error 41 us; skew 500 ppm
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:27.034376 17880 init.cc:377] Logged in from keytab as oryx/127.10.236.252@KRBTEST.COM (short username oryx)
I20260926 13:50:27.035374 17880 webserver.cc:460] Webserver started at http://127.10.236.252:38891/ using document root <none> and password file <none>
I20260926 13:50:27.036019 17880 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:27.036116 17880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:27.036358 17880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:27.038534 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/instance:
uuid: "7cfbadee761a4efd81b106fd90c1e717"
format_stamp: "Formatted at 2026-09-26 13:50:27 on dist-test-slave-r34j"
server_key: "43224af3f8617ff0311c8f417c3b6acb"
server_key_iv: "6d0b8c85710b9942bc6de187d2a7f55b"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.039114 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal/instance:
uuid: "7cfbadee761a4efd81b106fd90c1e717"
format_stamp: "Formatted at 2026-09-26 13:50:27 on dist-test-slave-r34j"
server_key: "43224af3f8617ff0311c8f417c3b6acb"
server_key_iv: "6d0b8c85710b9942bc6de187d2a7f55b"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.043171 17880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:27.046175 17979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.047659 17880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:27.047791 17880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
uuid: "7cfbadee761a4efd81b106fd90c1e717"
format_stamp: "Formatted at 2026-09-26 13:50:27 on dist-test-slave-r34j"
server_key: "43224af3f8617ff0311c8f417c3b6acb"
server_key_iv: "6d0b8c85710b9942bc6de187d2a7f55b"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.047901 17880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:27.066962 17880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:27.069727 17503 heartbeater.cc:503] Master 127.10.236.254:36389 was elected leader, sending a full tablet report...
I20260926 13:50:27.070570 17880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:27.070784 17880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:27.078675 17880 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.252:34915
I20260926 13:50:27.078976 18031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.252:34915 every 8 connection(s)
I20260926 13:50:27.079798 17880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
I20260926 13:50:27.082804 18032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:27.086419 18032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:50:27.087432 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 17880
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.252@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.236.252@KRBTEST.COM
I20260926 13:50:27.101560 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.088869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.10.236.254:36389)
0926 13:50:27.089004 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.089008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.089806 (+ 798us) server_negotiation.cc:407] Connection header received
0926 13:50:27.090442 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.090445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.090492 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.090589 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.091794 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.092635 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.094190 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.094536 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.094642 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.097555 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.097558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.097561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.097592 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.099104 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.099762 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.099767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.099769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.099857 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.100284 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.100287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.100288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.100540 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.100635 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.101241 (+ 606us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.101375 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:50:27.102474 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.089038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:42768)
0926 13:50:27.089636 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.089696 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.090242 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.090711 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.090717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.091232 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.091623 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.091663 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.092815 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.092818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.094060 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.094074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.094715 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.094718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.094926 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.095563 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.095586 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.097394 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.099221 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.099231 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.099242 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.099658 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.099961 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.099965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.099967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.100172 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.100651 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.100660 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.100996 (+ 336us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.101144 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.101497 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":48,"spinlock_wait_cycles":51584,"thread_start_us":147,"threads_started":1}
I20260926 13:50:27.102474 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.089038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.10.236.252:34915)
0926 13:50:27.089470 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.089483 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.089513 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:50:27.090221 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.090238 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.090467 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.090635 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.091665 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.092911 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.094185 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.094733 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.094890 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.098095 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.098105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.098120 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.098146 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.099962 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.100431 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.100436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.100438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.100528 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.100853 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.100855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.100857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.101079 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.101303 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.101533 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.101701 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":315,"spinlock_wait_cycles":3328,"thread_start_us":147,"threads_started":1}
I20260926 13:50:27.103785 18032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.252@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
I20260926 13:50:27.113395 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.104784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:49964)
0926 13:50:27.104916 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.104951 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.105031 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.105486 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.105488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.105618 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.106099 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.106106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.106978 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.106981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.107561 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.107570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.108003 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.108006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.108072 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.108547 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.108558 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.109817 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.111979 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.111982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.111983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.112166 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.112489 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.112492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.112493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.112586 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.112971 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.112973 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.113057 (+ 84us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.113149 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.113270 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 13:50:27.113540 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.105038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.10.236.253:39781)
0926 13:50:27.105291 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.105295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.105306 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:50:27.105341 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.105343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.105379 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.105450 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.106217 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.106883 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.107672 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.107906 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.107989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.110110 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.110113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.110115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.110137 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.111881 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.112286 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.112289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.112290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.112372 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.112693 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.112695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.112696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.112875 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.112953 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.113276 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.113382 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
I20260926 13:50:27.114217 18032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.252@KRBTEST.COM for oryx/127.10.236.252@KRBTEST.COM
I20260926 13:50:27.123438 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.114685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.0.0.1:60954)
0926 13:50:27.114852 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.114869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.114952 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.115209 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.115212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.115387 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.115461 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.115466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.116586 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.116589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.117508 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.117523 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.118247 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.118250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.118345 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.118914 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.118928 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.120373 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.121938 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.121942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.121945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.122135 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.122436 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.122439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.122441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.122519 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.122882 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.122885 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.122963 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.123081 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.123228 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 13:50:27.123438 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.114790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.10.236.252:34915)
0926 13:50:27.114911 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.114915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.114943 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:50:27.115064 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.115066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.115103 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.115199 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.115618 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.116452 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.117668 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.118129 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.118246 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.120513 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.120516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.120519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.120540 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.121799 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.122256 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.122260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.122261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.122335 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.122609 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.122614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.122616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.122787 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.122895 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.123122 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.123280 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":52}
I20260926 13:50:27.126904 18032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Bootstrap starting.
I20260926 13:50:27.129240 18032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:27.129989 18032 log.cc:824] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:27.132279 18032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: No bootstrap required, opened a new log
I20260926 13:50:27.135933 18032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } }
I20260926 13:50:27.136198 18032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:27.136236 18032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cfbadee761a4efd81b106fd90c1e717, State: Initialized, Role: FOLLOWER
I20260926 13:50:27.136843 18032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } }
I20260926 13:50:27.137106 18038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:27.137419 18038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } }
I20260926 13:50:27.138352 18038 leader_election.cc:288] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253:39781)
I20260926 13:50:27.138199 18039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } } }
I20260926 13:50:27.138459 18039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:27.139001 17924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cfbadee761a4efd81b106fd90c1e717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" is_pre_election: true
I20260926 13:50:27.138988 17322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cfbadee761a4efd81b106fd90c1e717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" is_pre_election: true
I20260926 13:50:27.139195 17924 raft_consensus.cc:1799] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Handling vote request from an unknown peer 7cfbadee761a4efd81b106fd90c1e717
I20260926 13:50:27.139319 17924 raft_consensus.cc:2396] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 7cfbadee761a4efd81b106fd90c1e717 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260926 13:50:27.139349 18032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:27.139931 17983 leader_election.cc:302] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cfbadee761a4efd81b106fd90c1e717; no voters: 4f95ea2a4aba495e9221893b3cc9aa92, fccd9ad787424bca8fa7f3645c0da8a6
I20260926 13:50:27.140231 18039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:27.141486 18039 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 13:50:27.145299 18050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:50:27.145416 18050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:50:27.148331 17880 master_runner.cc:428] Detected that this master 7cfbadee761a4efd81b106fd90c1e717 is joining an existing cluster
I20260926 13:50:27.148393 17880 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.236.252:34915
I20260926 13:50:27.156134 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17374
I20260926 13:50:27.161062 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.150714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.10.236.252:34915)
0926 13:50:27.150830 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.150834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.152048 (+ 1214us) server_negotiation.cc:407] Connection header received
0926 13:50:27.152198 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.152201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.152248 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.152297 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.153609 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.154354 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.155470 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.155818 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.155891 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.157682 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.157686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.157689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.157710 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.159440 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.159918 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.159921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.159923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.159991 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.160250 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.160253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.160254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.160445 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.160508 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.160786 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.160898 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:50:27.161981 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.150956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.0.0.1:60966)
0926 13:50:27.151968 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.151983 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.152106 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.152424 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.152427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.152776 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.153492 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.153502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.154474 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.154479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.155348 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.155358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.156026 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.156029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.156117 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.156975 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.156996 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.157550 (+ 554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.159587 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.159590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.159592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.159824 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.160084 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.160086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.160088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.160157 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.160561 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.160564 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.160633 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.161660 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.161807 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":1,"spinlock_wait_cycles":1920,"thread_start_us":92,"threads_started":1}
I20260926 13:50:27.164193 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:36001
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:36389
--webserver_port=40295
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:50:27.166138 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.150642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:42782)
0926 13:50:27.151847 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.151866 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.151989 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.152391 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.152394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.152750 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.152947 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.152956 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.154034 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.154037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.155064 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.155074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.160832 (+ 5758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.160839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.160975 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.161482 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.161501 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.161958 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.164120 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.164123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.164125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.164649 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.164985 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.164988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.164990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.165079 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.165578 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.165581 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.165700 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.165825 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.165965 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"mutex_wait_us":109,"spinlock_wait_cycles":1792,"thread_start_us":104,"threads_started":1}
I20260926 13:50:27.166139 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.150638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.10.236.254:36389)
0926 13:50:27.150863 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.150867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.152081 (+ 1214us) server_negotiation.cc:407] Connection header received
0926 13:50:27.152139 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.152145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.152199 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.152269 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.153086 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.153911 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.160083 (+ 6172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.160559 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.160687 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.162124 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.162131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.162137 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.162171 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.164011 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.164774 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.164780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.164782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.164882 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.165180 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.165186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.165191 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.165458 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.165554 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.165887 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.166003 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430625, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.254@KRBTEST.COM for oryx/127.10.236.252@KRBTEST.COM
I20260926 13:50:27.180301 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.151143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49972 (local address 127.10.236.253:39781)
0926 13:50:27.151252 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.151258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.152016 (+ 758us) server_negotiation.cc:407] Connection header received
0926 13:50:27.152173 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.152175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.152223 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.152276 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.153114 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.153903 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.155379 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.155720 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.155799 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.176569 (+ 20770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.176575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.176578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.176608 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.178407 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.178926 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.178929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.178931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.179008 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.179296 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.179299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.179301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.179540 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.179608 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.179950 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.180109 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:50:27.180989 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.170208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.10.236.252:34915)
0926 13:50:27.170569 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.170574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.170584 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:50:27.170648 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.170651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.170698 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.170764 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.171315 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.172110 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.173298 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.173632 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.173724 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.176592 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.176596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.176599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.176622 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.178689 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.179169 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.179172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.179174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.179278 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.179594 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.179597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.179599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.179844 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.179932 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.180664 (+ 732us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.180821 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"spinlock_wait_cycles":1408}
I20260926 13:50:27.180866 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.151087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:49972)
0926 13:50:27.151946 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.151957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.152074 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.152473 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.152475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.152821 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.152954 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.152969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.154034 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.154037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.155247 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.155256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.155980 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.155983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.156069 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.175907 (+ 19838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.175923 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.176373 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.178540 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.178543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.178546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.178815 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.179112 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.179114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.179116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.179194 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.179676 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.179678 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.179768 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.180539 (+ 771us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.180693 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":90,"thread_start_us":654,"threads_started":1}
I20260926 13:50:27.181664 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.170161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.0.0.1:60982)
0926 13:50:27.170342 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.170357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.170435 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.170878 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.170880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.171075 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.171178 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.171185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.172234 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.172299 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.173144 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.173155 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.173852 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.173856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.173951 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.174459 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.174474 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.176440 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.178815 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.178818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.178820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.179065 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.179376 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.179379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.179381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.179462 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.179976 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.179979 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.180110 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.180210 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.180918 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 13:50:27.183518 17312 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.236.252:34915: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717"
member_type: NON_VOTER
last_known_addr {
host: "127.10.236.252"
port: 34915
}
attrs {
promote: true
}
}
dest_uuid: "4f95ea2a4aba495e9221893b3cc9aa92"
cas_config_opid_index: 5
I20260926 13:50:27.184157 17312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } }
I20260926 13:50:27.185190 17872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.186756 17872 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.236.252:34915
I20260926 13:50:27.186924 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.187014 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.187922 17880 master.cc:537] Master@127.10.236.252:34915 shutting down...
I20260926 13:50:27.188206 18006 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Refusing update from remote peer 4f95ea2a4aba495e9221893b3cc9aa92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 13:50:27.188797 17875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:27.190147 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.185373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:49980)
0926 13:50:27.185641 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.185655 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.185751 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.186135 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.186142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.186159 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.186246 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.186252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.187369 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.187374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.188634 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.188646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.189732 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.189736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.189833 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.189884 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.189953 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.190026 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260926 13:50:27.190480 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.185463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.10.236.253:39781)
0926 13:50:27.185571 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.185575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.185721 (+ 146us) server_negotiation.cc:407] Connection header received
0926 13:50:27.185852 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.185855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.185912 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.186009 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.186371 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.187187 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.188762 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.189390 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.189488 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.189638 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.190301 (+ 663us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.190372 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260926 13:50:27.190534 17297 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 -> Peer 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252:34915): Couldn't send request to peer 7cfbadee761a4efd81b106fd90c1e717. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 13:50:27.190851 17924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Refusing update from remote peer 4f95ea2a4aba495e9221893b3cc9aa92: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 13:50:27.191262 17875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 13:50:27.193039 17924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.195326 17953 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f95ea2a4aba495e9221893b3cc9aa92. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.195446 17953 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:50:27.195595 17953 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: NON_VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.195664 17953 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:50:27.196666 17872 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: attempting to promote NON_VOTER fccd9ad787424bca8fa7f3645c0da8a6 to VOTER
I20260926 13:50:27.197086 17872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } }
W20260926 13:50:27.198769 17297 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 -> Peer 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252:34915): Couldn't send request to peer 7cfbadee761a4efd81b106fd90c1e717. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 13:50:27.199460 17924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 LEARNER]: Refusing update from remote peer 4f95ea2a4aba495e9221893b3cc9aa92: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 13:50:27.199855 17872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 13:50:27.201529 17875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.201925 17924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.203176 17875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.203270 17875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.203395 17875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.203454 17875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.204507 18061 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.204684 18061 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:27.205287 17880 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:27.205579 17880 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:27.205688 17880 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: stopping tablet replica
I20260926 13:50:27.210546 17880 master.cc:552] Master@127.10.236.252:34915 shutdown complete.
I20260926 13:50:27.210644 17880 master_runner.cc:305] Clearing existing system tablet
I20260926 13:50:27.210943 17880 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:27.211047 17880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:27.212435 17880 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:27.213414 18064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.213640 17880 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:27.213753 17880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:27.213785 17880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
uuid: "7cfbadee761a4efd81b106fd90c1e717"
format_stamp: "Formatted at 2026-09-26 13:50:27 on dist-test-slave-r34j"
server_key: "43224af3f8617ff0311c8f417c3b6acb"
server_key_iv: "6d0b8c85710b9942bc6de187d2a7f55b"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.215808 17880 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:27.218160 17880 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 13:50:27.218246 17880 log.cc:1196] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 13:50:27.218576 17880 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Deleting consensus metadata
I20260926 13:50:27.218747 17880 master_runner.cc:315] Copying system tablet from 127.10.236.254:36389
I20260926 13:50:27.219905 17880 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: tablet copy: Beginning tablet copy session from remote peer at address 127.10.236.254:36389
I20260926 13:50:27.229045 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.220451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:42790)
0926 13:50:27.220738 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.220754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.220869 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.221134 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.221137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.221387 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.221508 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.221515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.222587 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.222592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.223609 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.223624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.224227 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.224231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.224347 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.224954 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.224972 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.225422 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.227382 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.227385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.227388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.227711 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.227996 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.227999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.228001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.228091 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.228501 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.228504 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.228603 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.228708 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.228845 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260926 13:50:27.229403 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.220735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.10.236.254:36389)
0926 13:50:27.220858 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.220863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.220881 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:50:27.220959 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.220962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.221010 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.221099 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.221641 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.222453 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.223747 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.224084 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.224196 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.225566 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.225571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.225573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.225601 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.227268 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.227808 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.227812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.227814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.227900 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.228191 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.228194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.228196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.228405 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.228498 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.228776 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.228896 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:27.229867 17332 tablet_copy_service.cc:140] P 4f95ea2a4aba495e9221893b3cc9aa92: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7cfbadee761a4efd81b106fd90c1e717 ({username='oryx', principal='oryx/127.10.236.252@KRBTEST.COM'} at 127.0.0.1:42790)
I20260926 13:50:27.229970 17332 tablet_copy_service.cc:161] P 4f95ea2a4aba495e9221893b3cc9aa92: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7cfbadee761a4efd81b106fd90c1e717 at {username='oryx', principal='oryx/127.10.236.252@KRBTEST.COM'} at 127.0.0.1:42790: session id = 7cfbadee761a4efd81b106fd90c1e717-00000000000000000000000000000000
I20260926 13:50:27.231045 17332 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 13:50:27.232445 17880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 13:50:27.234766 17880 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: tablet copy: Starting download of 0 data blocks...
I20260926 13:50:27.234939 17880 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: tablet copy: Starting download of 1 WAL segments...
I20260926 13:50:27.236495 17880 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 13:50:27.238476 17332 tablet_copy_service.cc:342] P 4f95ea2a4aba495e9221893b3cc9aa92: Request end of tablet copy session 7cfbadee761a4efd81b106fd90c1e717-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.236.252@KRBTEST.COM'} at 127.0.0.1:42790
I20260926 13:50:27.238551 17332 tablet_copy_service.cc:433] P 4f95ea2a4aba495e9221893b3cc9aa92: ending tablet copy session 7cfbadee761a4efd81b106fd90c1e717-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7cfbadee761a4efd81b106fd90c1e717
I20260926 13:50:27.246265 17880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:27.248236 18071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.248445 18072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.248687 18074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:27.252727 17880 server_base.cc:1034] running on GCE node
I20260926 13:50:27.252938 17880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:27.253284 17880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:27.254451 17880 hybrid_clock.cc:648] HybridClock initialized: now 1790430627254441 us; error 25 us; skew 500 ppm
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:27.256157 17880 init.cc:377] Logged in from keytab as oryx/127.10.236.252@KRBTEST.COM (short username oryx)
I20260926 13:50:27.256672 17880 webserver.cc:460] Webserver started at http://127.10.236.252:33759/ using document root <none> and password file <none>
I20260926 13:50:27.256877 17880 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:27.256919 17880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:27.258262 17880 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:27.259305 18081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.259673 17880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:27.259791 17880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
uuid: "7cfbadee761a4efd81b106fd90c1e717"
format_stamp: "Formatted at 2026-09-26 13:50:27 on dist-test-slave-r34j"
server_key: "43224af3f8617ff0311c8f417c3b6acb"
server_key_iv: "6d0b8c85710b9942bc6de187d2a7f55b"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.259938 17880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:27.273821 17880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:27.276084 17880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:27.281395 17880 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.252:34915
I20260926 13:50:27.282153 17880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
I20260926 13:50:27.282662 18133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.252:34915 every 8 connection(s)
I20260926 13:50:27.284173 18134 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:50:27.285259 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Bootstrap starting.
I20260926 13:50:27.287245 17637 heartbeater.cc:503] Master 127.10.236.254:36389 was elected leader, sending a full tablet report...
I20260926 13:50:27.296779 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:50:27.297137 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Bootstrap complete.
I20260926 13:50:27.297753 18134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } }
I20260926 13:50:27.297907 18134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7cfbadee761a4efd81b106fd90c1e717, State: Initialized, Role: LEARNER
I20260926 13:50:27.298015 18134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } }
I20260926 13:50:27.298472 18134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:27.298905 18137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.299119 18137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:50:27.302675 18148 catalog_manager.cc:1259] Loaded cluster ID: 614021588dc141c18498e5ceaef77458
I20260926 13:50:27.302738 18148 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: loading cluster ID for follower catalog manager: success
I20260926 13:50:27.304911 18148 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: acquiring CA information for follower catalog manager: success
I20260926 13:50:27.306340 18148 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 13:50:27.309666 18059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:27.309937 18059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:27.310047 18059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:27.314147 18059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:27.314235 18059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:27.314349 18059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:27.319834 18059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:36001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=40295
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:27.320886 18059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:27.321882 18059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:27.331354 18156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.331972 18154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.331382 18153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:27.332135 18059 server_base.cc:1034] running on GCE node
I20260926 13:50:27.332551 18059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:27.333238 18059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:27.334422 18059 hybrid_clock.cc:648] HybridClock initialized: now 1790430627334397 us; error 38 us; skew 500 ppm
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:27.337400 18059 init.cc:377] Logged in from keytab as oryx/127.10.236.193@KRBTEST.COM (short username oryx)
I20260926 13:50:27.338310 18059 webserver.cc:460] Webserver started at http://127.10.236.193:40295/ using document root <none> and password file <none>
I20260926 13:50:27.338886 18059 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:27.338992 18059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:27.343078 18059 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:27.345417 18163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.346805 18059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:27.346983 18059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "9c08d8b4d0874f5e948507bb37f8ff38"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "f5b3b6a924633c508975cc88fa6edd7e"
server_key_iv: "28b8c6d20778fb0940fc288c5d3168ba"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.347612 18059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:27.364912 18059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:27.368096 18059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:27.368306 18059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:27.368913 18059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:27.369767 18059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:27.369843 18059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.369925 18059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:27.369982 18059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.380012 18059 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:36001
I20260926 13:50:27.380120 18276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:36001 every 8 connection(s)
I20260926 13:50:27.381177 18059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:27.381664 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 18059
I20260926 13:50:27.381876 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17508
I20260926 13:50:27.390621 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:36329
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:36389
--webserver_port=44897
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.252@KRBTEST.COM
I20260926 13:50:27.401372 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.388165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57879 (local address 127.10.236.253:39781)
0926 13:50:27.388303 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.388310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.388565 (+ 255us) server_negotiation.cc:407] Connection header received
0926 13:50:27.388660 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.388663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.388711 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.388811 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.389391 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.390222 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.391628 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.391978 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.392043 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.396225 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.396231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.396236 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.396275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.399005 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.399762 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.399767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.399771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.399875 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.400422 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.400425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.400427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.400676 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.400802 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.401091 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.401214 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:50:27.401796 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.383413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:53271 (local address 127.10.236.252:34915)
0926 13:50:27.390312 (+ 6899us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.390318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.390332 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:27.390386 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.390391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.390445 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.390991 (+ 546us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.391097 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.392120 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.393310 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.393711 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.393814 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.396391 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.396397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.396399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.396430 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.398189 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.398696 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.398701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.398703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.398853 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.399250 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.399253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.399255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.399723 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.400008 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.401545 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.401680 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6803,"thread_start_us":129,"threads_started":1}
I20260926 13:50:27.402324 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.383899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:50941 (local address 127.10.236.254:36389)
0926 13:50:27.384035 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.384040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.385553 (+ 1513us) server_negotiation.cc:407] Connection header received
0926 13:50:27.386951 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.386958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.387043 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.387141 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.389080 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.389637 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.391668 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.392079 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.392176 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.396168 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.396173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.396176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.396209 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.398006 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.398916 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.398921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.398924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.399011 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.399326 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.399331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.399336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.399560 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.399647 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.402067 (+ 2420us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.402185 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:50:27.402684 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.383908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:50941)
0926 13:50:27.385362 (+ 1454us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.385419 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.386673 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.387550 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.387561 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.388105 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.388915 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.388937 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.389810 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.389813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.391509 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.391522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.392367 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.392371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.392457 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.393239 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.393280 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.395965 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.398146 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.398152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.398169 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.398803 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.399114 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.399117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.399119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.399209 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.399717 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.399724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.401354 (+ 1630us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.401990 (+ 636us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.402129 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":135,"spinlock_wait_cycles":4736,"thread_start_us":118,"threads_started":1}
I20260926 13:50:27.403131 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.383909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.10.236.193:53271)
0926 13:50:27.387165 (+ 3256us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.387182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.387273 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.390576 (+ 3303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.390579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.390764 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.390871 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.390878 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.392237 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.392240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.393140 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.393152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.393834 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.393838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.393936 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.394454 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.394470 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.396237 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.398304 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.398306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.398308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.398613 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.398957 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.398962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.398967 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.399134 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.399875 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.399878 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.400068 (+ 190us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.400421 (+ 353us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.400787 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3096,"mutex_wait_us":44,"thread_start_us":132,"threads_started":1}
I20260926 13:50:27.403973 18277 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36389
I20260926 13:50:27.404080 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.387886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.10.236.193:57879)
0926 13:50:27.388463 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.388479 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.388580 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.388865 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.388871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.389089 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.389247 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.389257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.390365 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.390369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.391503 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.391513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.392243 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.392246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.392430 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.393320 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.393332 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.396034 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.399158 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.399164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.399169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.399622 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.400038 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.400044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.400048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.400164 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.400785 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.400789 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.400904 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.401019 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.401189 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":90,"spinlock_wait_cycles":3328,"thread_start_us":81,"threads_started":1}
I20260926 13:50:27.404220 18277 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.404429 18278 heartbeater.cc:347] Connected to a master server at 127.10.236.252:34915
I20260926 13:50:27.404518 18278 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.404651 18279 heartbeater.cc:347] Connected to a master server at 127.10.236.253:39781
I20260926 13:50:27.404721 18279 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.404839 18277 heartbeater.cc:511] Master 127.10.236.254:36389 requested a full tablet report, sending...
I20260926 13:50:27.404857 18278 heartbeater.cc:511] Master 127.10.236.252:34915 requested a full tablet report, sending...
I20260926 13:50:27.405990 17312 ts_manager.cc:194] Re-registered known tserver with Master: 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001)
I20260926 13:50:27.406313 18097 ts_manager.cc:194] Registered new tserver with Master: 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001)
I20260926 13:50:27.406512 17312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:50941
I20260926 13:50:27.404860 18279 heartbeater.cc:511] Master 127.10.236.253:39781 requested a full tablet report, sending...
I20260926 13:50:27.408094 17914 ts_manager.cc:194] Registered new tserver with Master: 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001)
W20260926 13:50:27.513022 18285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:27.513338 18285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:27.513415 18285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:27.517411 18285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:27.517513 18285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:27.517606 18285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:27.517520 17771 heartbeater.cc:503] Master 127.10.236.254:36389 was elected leader, sending a full tablet report...
I20260926 13:50:27.523238 18285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:36329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=44897
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:27.524400 18285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:27.525489 18285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:27.535792 18291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.535800 18290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:27.535858 18285 server_base.cc:1034] running on GCE node
W20260926 13:50:27.535800 18293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:27.536506 18285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:27.537146 18285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:27.538365 18285 hybrid_clock.cc:648] HybridClock initialized: now 1790430627538283 us; error 70 us; skew 500 ppm
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:27.541172 18285 init.cc:377] Logged in from keytab as oryx/127.10.236.194@KRBTEST.COM (short username oryx)
I20260926 13:50:27.542120 18285 webserver.cc:460] Webserver started at http://127.10.236.194:44897/ using document root <none> and password file <none>
I20260926 13:50:27.542739 18285 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:27.542851 18285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:27.546905 18285 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:27.549073 18300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.550436 18285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:27.550585 18285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "396ab5e82406cf7a5877fea707fc7d9f"
server_key_iv: "5a62a38bbfe180f69f1e880e7880b8da"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.551213 18285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:27.576686 18285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:27.580502 18285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:27.580763 18285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:27.581431 18285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:27.582440 18285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:27.582508 18285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.582574 18285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:27.582628 18285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.593834 18285 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:36329
I20260926 13:50:27.593854 18413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:36329 every 8 connection(s)
I20260926 13:50:27.594944 18285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:27.600641 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 18285
I20260926 13:50:27.600864 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17642
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:27.608361 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:42945
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:36389
--webserver_port=35695
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.252@KRBTEST.COM
I20260926 13:50:27.613559 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.597060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:48255 (local address 127.10.236.254:36389)
0926 13:50:27.597214 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.597220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.598098 (+ 878us) server_negotiation.cc:407] Connection header received
0926 13:50:27.599657 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.599660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.599726 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.599801 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.602744 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.603326 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.604763 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.605142 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.605243 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.608397 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.608400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.608403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.608435 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.610624 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.611313 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.611317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.611318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.611425 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.611905 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.611908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.611911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.612149 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.612243 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.613210 (+ 967us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.613354 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:50:27.613816 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.596970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:34307 (local address 127.10.236.252:34915)
0926 13:50:27.597122 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.597127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.598095 (+ 968us) server_negotiation.cc:407] Connection header received
0926 13:50:27.599576 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.599583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.599651 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.599762 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.604373 (+ 4611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.605242 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.606617 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.606976 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.607040 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.609972 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.609975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.609978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.610011 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.612008 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.612478 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.612482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.612483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.612571 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.612858 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.612861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.612864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.613062 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.613145 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.613541 (+ 396us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.613677 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:50:27.614516 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.597440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.10.236.194:34307)
0926 13:50:27.597927 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.597987 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.599221 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.599956 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.599965 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.600566 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.604207 (+ 3641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.604214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.605385 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.605387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.606464 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.606476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.607236 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.607246 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.607310 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.607868 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.607882 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.609798 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.612118 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.612121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.612124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.612366 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.612676 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.612679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.612681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.612755 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.613197 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.613201 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.613359 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.613944 (+ 585us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.614073 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":35,"spinlock_wait_cycles":30208,"thread_start_us":144,"threads_started":1}
I20260926 13:50:27.614806 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.597440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:48255)
0926 13:50:27.597927 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.597986 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.599429 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.599962 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.599965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.600608 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.601402 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.601427 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.603460 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.603467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.604601 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.604610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.605377 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.605381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.605490 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.606153 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.606168 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.608173 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.610725 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.610729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.610732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.611177 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.611579 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.611583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.611585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.611779 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.612297 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.612307 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.612721 (+ 414us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.612916 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.613915 (+ 999us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":811,"spinlock_wait_cycles":88448,"thread_start_us":151,"threads_started":1}
I20260926 13:50:27.615830 18415 heartbeater.cc:347] Connected to a master server at 127.10.236.252:34915
I20260926 13:50:27.615923 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.598745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:37479 (local address 127.10.236.253:39781)
0926 13:50:27.598861 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.598865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.598987 (+ 122us) server_negotiation.cc:407] Connection header received
0926 13:50:27.599541 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.599545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.599597 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.599653 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.601681 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.602445 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.604187 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.604543 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.604620 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.608381 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.608385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.608389 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.608420 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.610473 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.611293 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.611298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.611302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.611407 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.611867 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.611872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.611875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.612145 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.612245 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.615628 (+ 3383us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.615762 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:50:27.614534 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.598497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.10.236.194:37479)
0926 13:50:27.598877 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.598891 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.599221 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.599956 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.599965 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.600566 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.601530 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.601538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.602614 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.602617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.604051 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.604061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.604816 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.604820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.605003 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.605700 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.605720 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.608192 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.610614 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.610620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.610639 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.611166 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.611544 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.611549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.611553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.611752 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.612299 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.612306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.612722 (+ 416us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.612918 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.613301 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":545,"spinlock_wait_cycles":11136,"thread_start_us":207,"threads_started":1}
I20260926 13:50:27.616102 18415 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.616619 18416 heartbeater.cc:347] Connected to a master server at 127.10.236.253:39781
I20260926 13:50:27.616703 18416 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.616751 18415 heartbeater.cc:511] Master 127.10.236.252:34915 requested a full tablet report, sending...
I20260926 13:50:27.616839 18416 heartbeater.cc:511] Master 127.10.236.253:39781 requested a full tablet report, sending...
I20260926 13:50:27.617492 18414 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36389
I20260926 13:50:27.617568 18414 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.617717 18414 heartbeater.cc:511] Master 127.10.236.254:36389 requested a full tablet report, sending...
I20260926 13:50:27.617909 17914 ts_manager.cc:194] Registered new tserver with Master: 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329)
I20260926 13:50:27.617938 18097 ts_manager.cc:194] Registered new tserver with Master: 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329)
I20260926 13:50:27.618458 17312 ts_manager.cc:194] Re-registered known tserver with Master: 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329)
I20260926 13:50:27.619053 17312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:48255
I20260926 13:50:27.695611 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.691275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.0.0.1:60984)
0926 13:50:27.691585 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.691599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.691718 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.692131 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.692137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.692161 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.692273 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.692280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.693377 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.693379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.694150 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.694159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.695162 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.695165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.695236 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.695258 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.695317 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.695390 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":114,"threads_started":1}
I20260926 13:50:27.695698 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.691422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.10.236.252:34915)
0926 13:50:27.691610 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.691616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.691699 (+ 83us) server_negotiation.cc:407] Connection header received
0926 13:50:27.691829 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.691833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.691905 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.692034 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.692421 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.693231 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.694275 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.695020 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.695169 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.695344 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.695385 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.695529 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 13:50:27.696664 18108 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 13:50:27.697158 18137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f95ea2a4aba495e9221893b3cc9aa92. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.697312 18137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: This master's current role is: LEARNER
W20260926 13:50:27.729355 18421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:27.729622 18421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:27.729666 18421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:27.734102 18421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:27.734160 18421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:27.734221 18421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:27.739873 18421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:42945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=35695
--tserver_master_addrs=127.10.236.254:36389,127.10.236.253:39781,127.10.236.252:34915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:27.740954 18421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:27.741938 18421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:27.752055 18427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.752063 18430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:27.752079 18428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:27.752414 18421 server_base.cc:1034] running on GCE node
I20260926 13:50:27.752974 18421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:27.753589 18421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:27.754784 18421 hybrid_clock.cc:648] HybridClock initialized: now 1790430627754757 us; error 31 us; skew 500 ppm
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:27.757870 18421 init.cc:377] Logged in from keytab as oryx/127.10.236.195@KRBTEST.COM (short username oryx)
I20260926 13:50:27.758903 18421 webserver.cc:460] Webserver started at http://127.10.236.195:35695/ using document root <none> and password file <none>
I20260926 13:50:27.759616 18421 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:27.759698 18421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:27.764430 18421 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:27.766745 18437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.768177 18421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:50:27.768280 18421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "9f429a955862402981ded49a581ba755"
format_stamp: "Formatted at 2026-09-26 13:50:26 on dist-test-slave-r34j"
server_key: "2d93f812fdc7dd544dfaadc47bc2d8e2"
server_key_iv: "4a55e6907e5b34e8f3804f16c03c930a"
server_key_version: "encryptionkey@0"
I20260926 13:50:27.768880 18421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:27.777808 18421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:27.781046 18421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:27.781246 18421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:27.781877 18421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:27.782783 18421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:27.782853 18421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.782922 18421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:27.782971 18421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:27.792670 18421 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:42945
I20260926 13:50:27.792688 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:42945 every 8 connection(s)
I20260926 13:50:27.793718 18421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:27.797348 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 18421
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.252@KRBTEST.COM
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.253@KRBTEST.COM
Sep 26 13:50:27 dist-test-slave-r34j krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430627, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:27.811190 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.800418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49984 (local address 127.10.236.253:39781)
0926 13:50:27.800958 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.800962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.801327 (+ 365us) server_negotiation.cc:407] Connection header received
0926 13:50:27.801385 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.801389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.801446 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.801533 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.802400 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.803150 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804258 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.804608 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.805056 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.806919 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.806923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.806926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.806951 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.808749 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.809456 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.809460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.809462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.809544 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.809872 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.809875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.809877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.810105 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.810237 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.810849 (+ 612us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.810983 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":89,"threads_started":1}
I20260926 13:50:27.811777 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.799741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.10.236.254:36389)
0926 13:50:27.800292 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.800297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.800679 (+ 382us) server_negotiation.cc:407] Connection header received
0926 13:50:27.800741 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.800746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.800799 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.800929 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.801732 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.802533 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.803773 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.804137 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804227 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.806058 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.806061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.806064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.806090 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.808813 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.809424 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.809427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.809429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.809513 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.809912 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.809918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.809923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.810140 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.810540 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.811275 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.811562 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":90,"threads_started":1}
I20260926 13:50:27.816051 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.797495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:43955 (local address 127.10.236.253:39781)
0926 13:50:27.797635 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.797638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.798041 (+ 403us) server_negotiation.cc:407] Connection header received
0926 13:50:27.798409 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.798413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.798470 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.798523 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.805298 (+ 6775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.806108 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.807246 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.807646 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.807757 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.810728 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.810732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.810734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.810757 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.813613 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814116 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.814121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.814123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.814208 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814618 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.814624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.814629 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.814841 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.814916 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.815769 (+ 853us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.815890 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 13:50:27.816056 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.795645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:60871 (local address 127.10.236.252:34915)
0926 13:50:27.795779 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.795783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.797024 (+ 1241us) server_negotiation.cc:407] Connection header received
0926 13:50:27.798438 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.798445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.798512 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.798569 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.801782 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.802547 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804224 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.804578 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804675 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.809987 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.809991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.809993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.810021 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.813180 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814275 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.814281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.814284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.814383 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814662 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.814668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.814671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.814848 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.814931 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.815769 (+ 838us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.815892 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":52}
I20260926 13:50:27.817139 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.796223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:57513)
0926 13:50:27.796847 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.796908 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.798160 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.798857 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.798860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.799621 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.800428 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.800449 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.804470 (+ 4021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.805916 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.805925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.806632 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.806635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.806744 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.808708 (+ 1964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.808725 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.810951 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.813319 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.813327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.813355 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.814173 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.814854 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.814865 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.814960 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.815468 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.815471 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.815592 (+ 121us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.815706 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.816066 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":531,"spinlock_wait_cycles":20096,"thread_start_us":142,"threads_started":1}
I20260926 13:50:27.817166 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.797361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.10.236.195:43955)
0926 13:50:27.797946 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.797957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.798166 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.798848 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.798857 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.805036 (+ 6179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.805168 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.805175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.806254 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.806258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.807144 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.807156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.807918 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.807924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.808025 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.808466 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.808483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.810584 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.813739 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.813741 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.813743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.814007 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.814317 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.814321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.814511 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.814987 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.814993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.815448 (+ 455us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.816377 (+ 929us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.816526 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":47,"spinlock_wait_cycles":3328,"thread_start_us":72,"threads_started":1}
I20260926 13:50:27.817747 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.796223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.10.236.195:60871)
0926 13:50:27.796847 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.796908 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.798160 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.798848 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.798857 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.799621 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:27.801649 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.801659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.802699 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.802703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.804087 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.804097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.804819 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.805082 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.807036 (+ 1954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:27.807061 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:27.809786 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:27.813370 (+ 3584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.813375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.813378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.814164 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.814479 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814481 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:27.814483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:27.814563 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:27.814987 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:27.814993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:27.815448 (+ 455us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.816516 (+ 1068us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.816624 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":1720,"spinlock_wait_cycles":9600,"thread_start_us":144,"threads_started":1}
I20260926 13:50:27.818053 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.795645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:57513 (local address 127.10.236.254:36389)
0926 13:50:27.795778 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.795783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.797025 (+ 1242us) server_negotiation.cc:407] Connection header received
0926 13:50:27.798487 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.798493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.798551 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.798620 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.800596 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.801448 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.806037 (+ 4589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.806393 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.806506 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.811093 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.811098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.811102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.811130 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.813167 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.814507 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.814517 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.814522 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.814736 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.815069 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.815075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.815079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.815330 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.815422 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.817795 (+ 2373us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.817913 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:50:27.818687 18552 heartbeater.cc:347] Connected to a master server at 127.10.236.252:34915
I20260926 13:50:27.818946 18552 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.819010 18551 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36389
I20260926 13:50:27.819078 18551 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.819624 18552 heartbeater.cc:511] Master 127.10.236.252:34915 requested a full tablet report, sending...
I20260926 13:50:27.819624 18551 heartbeater.cc:511] Master 127.10.236.254:36389 requested a full tablet report, sending...
I20260926 13:50:27.816689 17312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:27.820636 17312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:27.820873 17311 ts_manager.cc:194] Re-registered known tserver with Master: 9f429a955862402981ded49a581ba755 (127.10.236.195:42945)
I20260926 13:50:27.820936 18097 ts_manager.cc:194] Registered new tserver with Master: 9f429a955862402981ded49a581ba755 (127.10.236.195:42945)
I20260926 13:50:27.821465 17311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:57513
I20260926 13:50:27.822772 18553 heartbeater.cc:347] Connected to a master server at 127.10.236.253:39781
I20260926 13:50:27.822867 18553 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:27.823348 18553 heartbeater.cc:511] Master 127.10.236.253:39781 requested a full tablet report, sending...
I20260926 13:50:27.823784 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: NON_VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: true } } }
I20260926 13:50:27.823937 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.824491 17914 ts_manager.cc:194] Registered new tserver with Master: 9f429a955862402981ded49a581ba755 (127.10.236.195:42945)
I20260926 13:50:27.827678 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.799528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.10.236.252:34915)
0926 13:50:27.800460 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.800472 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.800484 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:50:27.800528 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.800531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.800578 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.801914 (+ 1336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.802035 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.802845 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.804108 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.813895 (+ 9787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.814007 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.821941 (+ 7934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.821947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.823357 (+ 1410us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.823388 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:27.825191 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.825779 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.825782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.825783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.825870 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:27.826245 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:27.826249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:27.826251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:27.826495 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:27.827225 (+ 730us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.827304 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.827415 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9014,"server-negotiator.queue_time_us":862,"thread_start_us":81,"threads_started":1}
I20260926 13:50:27.831348 17872 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: attempting to promote NON_VOTER 7cfbadee761a4efd81b106fd90c1e717 to VOTER
I20260926 13:50:27.831764 17872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:27.833002 17924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Refusing update from remote peer 4f95ea2a4aba495e9221893b3cc9aa92: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 13:50:27.833425 18062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 13:50:27.833487 18108 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 LEARNER]: Refusing update from remote peer 4f95ea2a4aba495e9221893b3cc9aa92: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 13:50:27.833839 18062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 13:50:27.835932 17872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.836074 17924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.836232 18108 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.837441 18062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.837550 18062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.837489 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.837598 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.837690 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.837762 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:27.839295 18137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.839406 18137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:27.840372 18577 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:27.840507 18577 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:27.844559 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.839473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36329 (local address 127.0.0.1:40204)
0926 13:50:27.839734 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.839746 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.839844 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.841286 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.841288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.841304 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.841384 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.841389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.842281 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.842288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.843260 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.843268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.844169 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.844173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.844265 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.844287 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.844349 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.844420 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260926 13:50:27.844590 18581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.840512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.10.236.194:36329)
0926 13:50:27.840838 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.840846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.840865 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:50:27.840925 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.840929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.841137 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.841317 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.841465 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.842163 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.843396 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.844042 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.844183 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.844322 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.844384 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.844467 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":120,"threads_started":1}
I20260926 13:50:27.844842 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.839220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36001 (local address 127.0.0.1:54108)
0926 13:50:27.839366 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.839384 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.839459 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.840998 (+ 1539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.841001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.841019 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.841122 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.841128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.842457 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.842461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.843335 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.843344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.844463 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.844469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.844561 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.844590 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.844638 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.844709 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":52}
I20260926 13:50:27.845403 18580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.840130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54108 (local address 127.10.236.193:36001)
0926 13:50:27.840412 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.840425 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.840449 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:50:27.840539 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.840546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.840839 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.841019 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.841237 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.842332 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.843469 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.844307 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.845019 (+ 712us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.845141 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.845185 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.845263 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":120,"threads_started":1}
I20260926 13:50:27.847666 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.839828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:42945 (local address 127.0.0.1:59976)
0926 13:50:27.840059 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.840072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.840151 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.842435 (+ 2284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.842438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.842456 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.842611 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.842618 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.845483 (+ 2865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.845487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.846527 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.846536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.847296 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.847299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.847377 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.847397 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.847455 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.847544 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":40,"thread_start_us":103,"threads_started":1}
I20260926 13:50:27.847707 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.840862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.10.236.195:42945)
0926 13:50:27.842013 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.842023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.842044 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:50:27.842111 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.842116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.842297 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.842451 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.843537 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.845350 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.846651 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.847158 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.847303 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.847418 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.847501 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.847582 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":88,"threads_started":1}
I20260926 13:50:27.847682 18348 tablet_service.cc:1467] Processing CreateTablet for tablet eaa32f35921a432ca9e7a99e36742f98 (DEFAULT_TABLE table=test-table [id=d2b0faea688842b1b66f241a0affbc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:27.848009 18211 tablet_service.cc:1467] Processing CreateTablet for tablet eaa32f35921a432ca9e7a99e36742f98 (DEFAULT_TABLE table=test-table [id=d2b0faea688842b1b66f241a0affbc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:27.848809 18348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa32f35921a432ca9e7a99e36742f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:27.848819 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa32f35921a432ca9e7a99e36742f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:27.857404 18583 tablet_bootstrap.cc:492] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Bootstrap starting.
I20260926 13:50:27.858381 18584 tablet_bootstrap.cc:492] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Bootstrap starting.
I20260926 13:50:27.858577 18485 tablet_service.cc:1467] Processing CreateTablet for tablet eaa32f35921a432ca9e7a99e36742f98 (DEFAULT_TABLE table=test-table [id=d2b0faea688842b1b66f241a0affbc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:27.859867 18485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaa32f35921a432ca9e7a99e36742f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:27.860042 18583 tablet_bootstrap.cc:654] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:27.860605 18584 tablet_bootstrap.cc:654] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:27.861002 18583 log.cc:824] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:27.861479 18584 log.cc:824] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:27.863199 18583 tablet_bootstrap.cc:492] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: No bootstrap required, opened a new log
I20260926 13:50:27.863415 18583 ts_tablet_manager.cc:1397] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:50:27.867187 18584 tablet_bootstrap.cc:492] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: No bootstrap required, opened a new log
I20260926 13:50:27.867465 18584 ts_tablet_manager.cc:1397] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260926 13:50:27.868497 18587 tablet_bootstrap.cc:492] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Bootstrap starting.
I20260926 13:50:27.868317 18583 raft_consensus.cc:348] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.868597 18583 raft_consensus.cc:374] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:27.868670 18583 raft_consensus.cc:729] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c08d8b4d0874f5e948507bb37f8ff38, State: Initialized, Role: FOLLOWER
I20260926 13:50:27.869184 18583 consensus_queue.cc:260] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.870057 18277 heartbeater.cc:503] Master 127.10.236.254:36389 was elected leader, sending a full tablet report...
I20260926 13:50:27.870394 18583 ts_tablet_manager.cc:1428] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260926 13:50:27.870958 18587 tablet_bootstrap.cc:654] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:27.871318 18584 raft_consensus.cc:348] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.871644 18584 raft_consensus.cc:374] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:27.871706 18584 raft_consensus.cc:729] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5e28e1cedb4b0c9f47d46deffa2ff7, State: Initialized, Role: FOLLOWER
I20260926 13:50:27.872164 18584 consensus_queue.cc:260] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.873389 18584 ts_tablet_manager.cc:1428] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260926 13:50:27.873741 18587 log.cc:824] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:27.873796 18414 heartbeater.cc:503] Master 127.10.236.254:36389 was elected leader, sending a full tablet report...
I20260926 13:50:27.877113 18587 tablet_bootstrap.cc:492] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: No bootstrap required, opened a new log
I20260926 13:50:27.877328 18587 ts_tablet_manager.cc:1397] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260926 13:50:27.881239 18587 raft_consensus.cc:348] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.881475 18587 raft_consensus.cc:374] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:27.881551 18587 raft_consensus.cc:729] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f429a955862402981ded49a581ba755, State: Initialized, Role: FOLLOWER
I20260926 13:50:27.882135 18587 consensus_queue.cc:260] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.882896 18551 heartbeater.cc:503] Master 127.10.236.254:36389 was elected leader, sending a full tablet report...
I20260926 13:50:27.883255 18587 ts_tablet_manager.cc:1428] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260926 13:50:27.887928 18280 tablet.cc:2369] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:27.901611 18588 raft_consensus.cc:482] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:27.901831 18588 raft_consensus.cc:504] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.903055 18588 leader_election.cc:288] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329), 9f429a955862402981ded49a581ba755 (127.10.236.195:42945)
I20260926 13:50:27.907020 18581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.903410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:32851 (local address 127.10.236.194:36329)
0926 13:50:27.903767 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.903771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.903781 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:50:27.903824 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.903826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.903915 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.903990 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.904327 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.904813 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.906025 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.906409 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.906503 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.906543 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.906795 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.906873 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260926 13:50:27.907013 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.903224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36329 (local address 127.10.236.193:32851)
0926 13:50:27.903416 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.903433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.903551 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.904038 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.904041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.904058 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.904182 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.904188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.905029 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.905032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.905851 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.905860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.906540 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.906544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.906653 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.906687 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.906770 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.906852 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":5}
I20260926 13:50:27.907207 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.903537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:52949 (local address 127.10.236.195:42945)
0926 13:50:27.903676 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.903679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.903693 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:27.903742 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.903744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.903804 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.903872 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:27.904224 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.904762 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.905987 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.906595 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.906717 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.906780 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.906999 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.907088 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 13:50:27.907225 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.903383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:42945 (local address 127.10.236.193:52949)
0926 13:50:27.903539 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:27.903551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:27.903611 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:27.903929 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:27.903934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:27.903958 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:27.904103 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.904109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.904946 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.904952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.905830 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:27.905838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.906714 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.906716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:27.906862 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.906885 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:50:27.906962 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:50:27.907025 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":47}
I20260926 13:50:27.907872 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa32f35921a432ca9e7a99e36742f98" candidate_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f429a955862402981ded49a581ba755" is_pre_election: true
I20260926 13:50:27.908113 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa32f35921a432ca9e7a99e36742f98" candidate_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" is_pre_election: true
I20260926 13:50:27.908249 18505 raft_consensus.cc:2454] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c08d8b4d0874f5e948507bb37f8ff38 in term 0.
I20260926 13:50:27.908360 18368 raft_consensus.cc:2454] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c08d8b4d0874f5e948507bb37f8ff38 in term 0.
I20260926 13:50:27.908792 18165 leader_election.cc:302] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8b5e28e1cedb4b0c9f47d46deffa2ff7, 9c08d8b4d0874f5e948507bb37f8ff38, 9f429a955862402981ded49a581ba755; no voters:
I20260926 13:50:27.909082 18588 raft_consensus.cc:2781] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:27.909161 18588 raft_consensus.cc:482] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:27.909192 18588 raft_consensus.cc:3037] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:27.910312 18588 raft_consensus.cc:504] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.910676 18588 leader_election.cc:288] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329), 9f429a955862402981ded49a581ba755 (127.10.236.195:42945)
I20260926 13:50:27.911127 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa32f35921a432ca9e7a99e36742f98" candidate_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f429a955862402981ded49a581ba755"
I20260926 13:50:27.911149 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaa32f35921a432ca9e7a99e36742f98" candidate_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7"
I20260926 13:50:27.911278 18368 raft_consensus.cc:3037] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:27.911278 18505 raft_consensus.cc:3037] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:27.912735 18368 raft_consensus.cc:2454] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c08d8b4d0874f5e948507bb37f8ff38 in term 1.
I20260926 13:50:27.913127 18165 leader_election.cc:302] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5e28e1cedb4b0c9f47d46deffa2ff7, 9c08d8b4d0874f5e948507bb37f8ff38; no voters:
I20260926 13:50:27.913213 18505 raft_consensus.cc:2454] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c08d8b4d0874f5e948507bb37f8ff38 in term 1.
I20260926 13:50:27.913331 18588 raft_consensus.cc:2781] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:27.913576 18588 raft_consensus.cc:686] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 LEADER]: Becoming Leader. State: Replica: 9c08d8b4d0874f5e948507bb37f8ff38, State: Running, Role: LEADER
I20260926 13:50:27.913964 18588 consensus_queue.cc:237] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } }
I20260926 13:50:27.917804 17311 catalog_manager.cc:5495] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193). New cstate: current_term: 1 leader_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:27.941900 18580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.937971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54124 (local address 127.10.236.193:36001)
0926 13:50:27.938101 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.938105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.938119 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:27.938265 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.938267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.938319 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.938455 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:27.938761 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.939299 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.940302 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.940514 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.940619 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.940974 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:27.941425 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:27.941526 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:27.941677 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 13:50:27.941737 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:27.948195 18505 raft_consensus.cc:1261] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Refusing update from remote peer 9c08d8b4d0874f5e948507bb37f8ff38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:27.948195 18368 raft_consensus.cc:1261] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Refusing update from remote peer 9c08d8b4d0874f5e948507bb37f8ff38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:27.948988 18588 consensus_queue.cc:1035] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:27.949178 18592 consensus_queue.cc:1035] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:27.958011 18595 mvcc.cc:205] Tried to move back new op lower bound from 7333603852067602432 to 7333603851938738176. Current Snapshot: MvccSnapshot[applied={T|T < 7333603852067602432}]
I20260926 13:50:27.960330 18596 mvcc.cc:205] Tried to move back new op lower bound from 7333603852067602432 to 7333603851938738176. Current Snapshot: MvccSnapshot[applied={T|T < 7333603852067602432}]
I20260926 13:50:27.960457 18597 mvcc.cc:205] Tried to move back new op lower bound from 7333603852067602432 to 7333603851938738176. Current Snapshot: MvccSnapshot[applied={T|T < 7333603852067602432}]
I20260926 13:50:27.971076 17311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:27.971310 17311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:27.977062 17311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f95ea2a4aba495e9221893b3cc9aa92: Sending DeleteTablet for 3 replicas of tablet eaa32f35921a432ca9e7a99e36742f98
I20260926 13:50:27.977865 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet eaa32f35921a432ca9e7a99e36742f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:27 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:54108
I20260926 13:50:27.978071 18485 tablet_service.cc:1514] Processing DeleteTablet for tablet eaa32f35921a432ca9e7a99e36742f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:27 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:59976
I20260926 13:50:27.978070 18348 tablet_service.cc:1514] Processing DeleteTablet for tablet eaa32f35921a432ca9e7a99e36742f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:27 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:40204
I20260926 13:50:27.978624 18605 tablet_replica.cc:330] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: stopping tablet replica
I20260926 13:50:27.979032 18605 raft_consensus.cc:2229] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:27.979506 18605 raft_consensus.cc:2258] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:27.979737 18604 tablet_replica.cc:330] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: stopping tablet replica
I20260926 13:50:27.980065 18606 tablet_replica.cc:330] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: stopping tablet replica
I20260926 13:50:27.980093 18604 raft_consensus.cc:2229] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:27.980455 18606 raft_consensus.cc:2229] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:27.980521 18604 raft_consensus.cc:2258] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:27.981031 18606 raft_consensus.cc:2258] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:27.982103 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17280
I20260926 13:50:27.983707 18606 ts_tablet_manager.cc:1905] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:27.983712 18605 ts_tablet_manager.cc:1905] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:27.986491 18605 ts_tablet_manager.cc:1918] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:27.986491 18606 ts_tablet_manager.cc:1918] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:27.986603 18606 log.cc:1196] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/eaa32f35921a432ca9e7a99e36742f98
I20260926 13:50:27.986603 18605 log.cc:1196] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/eaa32f35921a432ca9e7a99e36742f98
I20260926 13:50:27.986948 18606 ts_tablet_manager.cc:1939] T eaa32f35921a432ca9e7a99e36742f98 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Deleting consensus metadata
I20260926 13:50:27.986948 18605 ts_tablet_manager.cc:1939] T eaa32f35921a432ca9e7a99e36742f98 P 9f429a955862402981ded49a581ba755: Deleting consensus metadata
I20260926 13:50:27.988855 18604 ts_tablet_manager.cc:1905] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:27.990972 18604 ts_tablet_manager.cc:1918] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:27.991096 18604 log.cc:1196] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/eaa32f35921a432ca9e7a99e36742f98
W20260926 13:50:27.991254 18301 connection.cc:535] server connection from 127.0.0.1:40204 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:50:27.992053 18604 ts_tablet_manager.cc:1939] T eaa32f35921a432ca9e7a99e36742f98 P 9c08d8b4d0874f5e948507bb37f8ff38: Deleting consensus metadata
W20260926 13:50:27.993175 18164 connection.cc:441] server connection from 127.0.0.1:54108 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54108 (request call id 1) could send its response
I20260926 13:50:28.003127 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.980853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49990 (local address 127.10.236.253:39781)
0926 13:50:27.981009 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.981015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.992318 (+ 11303us) server_negotiation.cc:407] Connection header received
0926 13:50:27.992456 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.992460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.992514 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.992597 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.993706 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.994427 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.995637 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.996022 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.996135 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:27.998064 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:27.998069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:27.998071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:27.998097 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:28.000660 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.001189 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.001195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.001199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.001294 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.001594 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.001598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.001599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.001817 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:28.001947 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:28.002824 (+ 877us) server_negotiation.cc:299] Negotiation successful
0926 13:50:28.002942 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":65}
I20260926 13:50:28.005016 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.995012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.10.236.253:39781)
0926 13:50:27.995134 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.995137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.995661 (+ 524us) server_negotiation.cc:407] Connection header received
0926 13:50:27.995824 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.995827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.995871 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.995918 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.996825 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.997602 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.998814 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.999185 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.999269 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:28.000961 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:28.000965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:28.000968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:28.000992 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:28.002822 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.003633 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.003636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.003638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.003718 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.004058 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.004064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.004068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.004286 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:28.004413 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:28.004729 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 13:50:28.004853 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":1280}
I20260926 13:50:28.005232 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.994723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.10.236.252:34915)
0926 13:50:27.994868 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.994873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:27.995389 (+ 516us) server_negotiation.cc:407] Connection header received
0926 13:50:27.995607 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:27.995611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:27.995681 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:27.995733 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:27.997073 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.997938 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.999280 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:27.999713 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:27.999823 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:28.001634 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:28.001641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:28.001656 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:28.001684 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:28.003352 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.003870 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.003873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.003875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.003986 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.004320 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.004323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.004325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.004535 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:28.004639 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:28.004977 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 13:50:28.005085 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:28.011788 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:27.980842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.10.236.252:34915)
0926 13:50:27.980953 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:27.980958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:28.002718 (+ 21760us) server_negotiation.cc:407] Connection header received
0926 13:50:28.003640 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:28.003643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:28.003696 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:28.003750 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:28.004638 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:28.005399 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:28.006439 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:28.006729 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:28.006789 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:28.008295 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:28.008299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:28.008302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:28.008328 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:28.009961 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.010431 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.010434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.010436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.010521 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:28.010865 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:28.010869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:28.010871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:28.011094 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:28.011208 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:28.011475 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 13:50:28.011670 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 13:50:28.963042 18615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:28.962529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:53153)
0926 13:50:28.962810 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:28.962916 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
W20260926 13:50:28.963665 18414 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:36389 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:28.964470 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:28.964001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:36177)
0926 13:50:28.964221 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:28.964322 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
W20260926 13:50:28.964798 18551 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:36389 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:28.965128 18617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:28.964749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:57141)
0926 13:50:28.964976 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:28.965055 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
W20260926 13:50:28.965374 18277 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:36389 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:29.477236 18618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f95ea2a4aba495e9221893b3cc9aa92)
I20260926 13:50:29.477383 18618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:29.477612 18619 raft_consensus.cc:482] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f95ea2a4aba495e9221893b3cc9aa92)
I20260926 13:50:29.477859 18619 raft_consensus.cc:504] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:29.478024 18618 leader_election.cc:288] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253:39781)
I20260926 13:50:29.478824 18619 leader_election.cc:288] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252:34915)
I20260926 13:50:29.479283 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.478191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43296)
0926 13:50:29.478731 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.479178 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":45,"threads_started":1}
I20260926 13:50:29.479285 18623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.478879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43306)
0926 13:50:29.479087 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.479155 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
W20260926 13:50:29.479451 17901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
W20260926 13:50:29.479584 18085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:29.480428 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.480192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43322)
0926 13:50:29.480314 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.480365 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 13:50:29.480600 18085 leader_election.cc:334] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:29.480844 18623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.480413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43324)
0926 13:50:29.480540 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.480782 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 13:50:29.480934 17901 leader_election.cc:334] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:29.483654 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.478994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:34915 (local address 127.0.0.1:32798)
0926 13:50:29.479329 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.479343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:29.479449 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:29.480006 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:29.480008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:29.480024 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:29.480155 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:29.480162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.481105 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.481108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:29.482294 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:29.482305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.483204 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.483208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:29.483301 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:29.483326 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 13:50:29.483404 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:50:29.483511 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":5,"thread_start_us":43,"threads_started":1}
I20260926 13:50:29.483695 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.479144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.10.236.252:34915)
0926 13:50:29.479658 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:29.479665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:29.479680 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:29.479742 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:29.479745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:29.479815 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:29.479897 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:29.480276 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.480998 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.482456 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.483071 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.483202 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:29.483253 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:29.483438 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 13:50:29.483561 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":448,"spinlock_wait_cycles":14848,"thread_start_us":46,"threads_started":1}
I20260926 13:50:29.483718 18621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.478548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:39781 (local address 127.0.0.1:50006)
0926 13:50:29.479145 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.479160 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:29.479302 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:29.479743 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:29.479747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:29.479792 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:29.479897 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:29.479907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.480996 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.481001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:29.482294 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:29.482305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.483204 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.483208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:29.483301 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:29.483326 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 13:50:29.483459 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 13:50:29.483571 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":2,"spinlock_wait_cycles":8960,"thread_start_us":54,"threads_started":1}
I20260926 13:50:29.483690 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.478648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.10.236.253:39781)
0926 13:50:29.478963 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:29.478968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:29.479224 (+ 256us) server_negotiation.cc:407] Connection header received
0926 13:50:29.479448 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:29.479454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:29.479535 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:29.479594 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:29.480045 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.480849 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.482456 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:29.483071 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:29.483202 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:29.483253 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:29.483472 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 13:50:29.483564 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":240,"thread_start_us":71,"threads_started":1}
I20260926 13:50:29.484248 17924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cfbadee761a4efd81b106fd90c1e717" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" is_pre_election: true
I20260926 13:50:29.484309 18108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7cfbadee761a4efd81b106fd90c1e717" is_pre_election: true
I20260926 13:50:29.484411 17924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cfbadee761a4efd81b106fd90c1e717 in term 1.
I20260926 13:50:29.484534 18108 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fccd9ad787424bca8fa7f3645c0da8a6 in term 1.
I20260926 13:50:29.484735 18085 leader_election.cc:302] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cfbadee761a4efd81b106fd90c1e717, fccd9ad787424bca8fa7f3645c0da8a6; no voters: 4f95ea2a4aba495e9221893b3cc9aa92
I20260926 13:50:29.484916 18618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 13:50:29.484874 17899 leader_election.cc:302] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cfbadee761a4efd81b106fd90c1e717, fccd9ad787424bca8fa7f3645c0da8a6; no voters: 4f95ea2a4aba495e9221893b3cc9aa92
I20260926 13:50:29.484972 18618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f95ea2a4aba495e9221893b3cc9aa92)
I20260926 13:50:29.485014 18618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:50:29.485116 18619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 13:50:29.485173 18619 raft_consensus.cc:482] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f95ea2a4aba495e9221893b3cc9aa92)
I20260926 13:50:29.485210 18619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:50:29.486119 18618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:29.486135 18619 raft_consensus.cc:504] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:29.486492 18618 leader_election.cc:288] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 2 election: Requested vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253:39781)
I20260926 13:50:29.486550 18619 leader_election.cc:288] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), 7cfbadee761a4efd81b106fd90c1e717 (127.10.236.252:34915)
I20260926 13:50:29.486802 18621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.486594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43334)
0926 13:50:29.486679 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.486736 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 13:50:29.486858 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.486626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43340)
0926 13:50:29.486748 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.486797 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 13:50:29.486949 18108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7cfbadee761a4efd81b106fd90c1e717"
I20260926 13:50:29.486948 17924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cfbadee761a4efd81b106fd90c1e717" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
I20260926 13:50:29.487083 17924 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cfbadee761a4efd81b106fd90c1e717 in current term 2: Already voted for candidate fccd9ad787424bca8fa7f3645c0da8a6 in this term.
I20260926 13:50:29.487092 18108 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fccd9ad787424bca8fa7f3645c0da8a6 in current term 2: Already voted for candidate 7cfbadee761a4efd81b106fd90c1e717 in this term.
I20260926 13:50:29.487582 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.487335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43352)
0926 13:50:29.487454 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.487526 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 13:50:29.487691 17901 leader_election.cc:334] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:29.487753 17901 leader_election.cc:302] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fccd9ad787424bca8fa7f3645c0da8a6; no voters: 4f95ea2a4aba495e9221893b3cc9aa92, 7cfbadee761a4efd81b106fd90c1e717
I20260926 13:50:29.487892 18619 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 13:50:29.487957 18621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.487711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43366)
0926 13:50:29.487843 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.487891 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260926 13:50:29.488057 18085 leader_election.cc:334] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:29.488114 18085 leader_election.cc:302] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cfbadee761a4efd81b106fd90c1e717; no voters: 4f95ea2a4aba495e9221893b3cc9aa92, fccd9ad787424bca8fa7f3645c0da8a6
I20260926 13:50:29.488258 18618 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 13:50:29.965183 18628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.964661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:59363)
0926 13:50:29.964973 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.965049 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":92,"threads_started":1}
I20260926 13:50:29.965704 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.965303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:42837)
0926 13:50:29.965531 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.965584 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260926 13:50:29.966204 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:29.965853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:51877)
0926 13:50:29.966082 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:29.966131 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
I20260926 13:50:30.966447 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:30.966024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:60203)
0926 13:50:30.966283 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:30.966355 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260926 13:50:30.966789 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:30.966374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:60785)
0926 13:50:30.966626 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:30.966687 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260926 13:50:30.967300 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:30.966806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:60611)
0926 13:50:30.967171 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:30.967224 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":58,"threads_started":1}
I20260926 13:50:31.187877 18634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:31.188019 18634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:31.188618 18634 leader_election.cc:288] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253:39781)
I20260926 13:50:31.189353 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.188832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43644)
0926 13:50:31.189128 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.189220 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":84,"threads_started":1}
I20260926 13:50:31.189433 17924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cfbadee761a4efd81b106fd90c1e717" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" is_pre_election: true
I20260926 13:50:31.189605 17924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cfbadee761a4efd81b106fd90c1e717 in term 2.
I20260926 13:50:31.189903 18085 leader_election.cc:302] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cfbadee761a4efd81b106fd90c1e717, fccd9ad787424bca8fa7f3645c0da8a6; no voters:
I20260926 13:50:31.190073 18634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 13:50:31.190142 18634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:31.190184 18634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 2 FOLLOWER]: Advancing to term 3
I20260926 13:50:31.190439 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.190179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43648)
0926 13:50:31.190325 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.190381 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 13:50:31.190639 18085 leader_election.cc:334] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:31.191357 18634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:31.191751 18634 leader_election.cc:288] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 3 election: Requested vote from peers 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389), fccd9ad787424bca8fa7f3645c0da8a6 (127.10.236.253:39781)
I20260926 13:50:31.192035 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.191827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43658)
0926 13:50:31.191914 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.191971 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 13:50:31.192221 17924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cfbadee761a4efd81b106fd90c1e717" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fccd9ad787424bca8fa7f3645c0da8a6"
I20260926 13:50:31.192344 17924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 13:50:31.192731 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.192521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43662)
0926 13:50:31.192626 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.192677 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:50:31.192847 18085 leader_election.cc:334] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
I20260926 13:50:31.193544 17924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cfbadee761a4efd81b106fd90c1e717 in term 3.
I20260926 13:50:31.193838 18085 leader_election.cc:302] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cfbadee761a4efd81b106fd90c1e717, fccd9ad787424bca8fa7f3645c0da8a6; no voters: 4f95ea2a4aba495e9221893b3cc9aa92
I20260926 13:50:31.194044 18634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 13:50:31.194227 18634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [term 3 LEADER]: Becoming Leader. State: Replica: 7cfbadee761a4efd81b106fd90c1e717, State: Running, Role: LEADER
I20260926 13:50:31.194515 18634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } }
I20260926 13:50:31.195822 18637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cfbadee761a4efd81b106fd90c1e717. Latest consensus state: current_term: 3 leader_uuid: "7cfbadee761a4efd81b106fd90c1e717" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:31.195953 18637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:31.196257 18639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:31.198621 18639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:31.199065 18639 catalog_manager.cc:1259] Loaded cluster ID: 614021588dc141c18498e5ceaef77458
I20260926 13:50:31.199108 18639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:31.199726 18639 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:31.200145 18639 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Loaded TSK: 0
I20260926 13:50:31.200821 18639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:31.233868 18095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:31.237028 18095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:31.240341 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.239979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43684)
0926 13:50:31.240174 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.240247 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 13:50:31.240630 17924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [term 3 FOLLOWER]: Refusing update from remote peer 7cfbadee761a4efd81b106fd90c1e717: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260926 13:50:31.241236 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.240985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.0.0.1:43700)
0926 13:50:31.241108 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.241164 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 13:50:31.241282 18634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 13:50:31.241560 18085 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 -> Peer 4f95ea2a4aba495e9221893b3cc9aa92 (127.10.236.254:36389): Couldn't send request to peer 4f95ea2a4aba495e9221893b3cc9aa92. Status: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 13:50:31.242862 18641 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cfbadee761a4efd81b106fd90c1e717. Latest consensus state: current_term: 3 leader_uuid: "7cfbadee761a4efd81b106fd90c1e717" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:31.243026 18641 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:31.244184 18637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7cfbadee761a4efd81b106fd90c1e717" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:31.244315 18637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:31.244686 18642 mvcc.cc:205] Tried to move back new op lower bound from 7333603865554051072 to 7333603865375125504. Current Snapshot: MvccSnapshot[applied={T|T < 7333603865554051072}]
I20260926 13:50:31.244848 18641 sys_catalog.cc:455] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7cfbadee761a4efd81b106fd90c1e717" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4f95ea2a4aba495e9221893b3cc9aa92" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36389 } } peers { permanent_uuid: "fccd9ad787424bca8fa7f3645c0da8a6" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "7cfbadee761a4efd81b106fd90c1e717" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 34915 } attrs { promote: false } } }
I20260926 13:50:31.244969 18641 sys_catalog.cc:458] T 00000000000000000000000000000000 P fccd9ad787424bca8fa7f3645c0da8a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:50:31.255849 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.250387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36329 (local address 127.0.0.1:40212)
0926 13:50:31.250499 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.250516 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:31.250618 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:31.251687 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:31.251691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:31.251711 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:31.251847 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.251854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.253070 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.253074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.254258 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.254271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.255334 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.255339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.255466 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.255544 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 13:50:31.255630 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 13:50:31.255703 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":141}
I20260926 13:50:31.255851 18647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.250546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.10.236.194:36329)
0926 13:50:31.251144 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:31.251149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:31.251164 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:31.251221 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:31.251225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:31.251276 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:31.251656 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:31.251986 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.252944 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.254540 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.255160 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.255293 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.255348 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:31.255647 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 13:50:31.255716 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":106,"threads_started":1}
I20260926 13:50:31.256177 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.251014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36001 (local address 127.0.0.1:54126)
0926 13:50:31.251243 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.251257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:31.251339 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:31.251960 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:31.251966 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:31.251982 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:31.252073 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.252079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.253218 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.253224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.254648 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.254660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.255771 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.255774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.255876 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.255902 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:50:31.255971 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:50:31.256027 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":24,"thread_start_us":98,"threads_started":1}
I20260926 13:50:31.256175 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.250651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:42945 (local address 127.0.0.1:59986)
0926 13:50:31.250908 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.250923 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:31.251011 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:31.252291 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:31.252294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:31.252312 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:31.252415 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.252422 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.253293 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.253296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.254682 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.254692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.255715 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.255719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.255823 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.255847 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:50:31.255916 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:50:31.256002 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":26,"thread_start_us":92,"threads_started":1}
I20260926 13:50:31.256561 18348 tablet_service.cc:1467] Processing CreateTablet for tablet 57aeefc6797e44389645dcd3b8026ab2 (DEFAULT_TABLE table=test-table [id=dadd84abc95f483893aa710fb7282f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:31.256655 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.250950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.10.236.195:42945)
0926 13:50:31.252010 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:31.252016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:31.252031 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:31.252088 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:31.252090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:31.252130 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:31.252285 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:31.252546 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.253185 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.254818 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.255522 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.256280 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.256351 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:31.256395 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 13:50:31.256491 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":84,"threads_started":1}
I20260926 13:50:31.256879 18348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57aeefc6797e44389645dcd3b8026ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:31.256963 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.251288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.10.236.193:36001)
0926 13:50:31.251603 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:31.251607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:31.251620 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:50:31.251690 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:31.251693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:31.251740 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:31.251830 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:31.252205 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.253095 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.254805 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.255524 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.255658 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.255722 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:31.256716 (+ 994us) server_negotiation.cc:299] Negotiation successful
0926 13:50:31.256822 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":74,"threads_started":1}
I20260926 13:50:31.257148 18485 tablet_service.cc:1467] Processing CreateTablet for tablet 57aeefc6797e44389645dcd3b8026ab2 (DEFAULT_TABLE table=test-table [id=dadd84abc95f483893aa710fb7282f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:31.257408 18485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57aeefc6797e44389645dcd3b8026ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:31.257442 18211 tablet_service.cc:1467] Processing CreateTablet for tablet 57aeefc6797e44389645dcd3b8026ab2 (DEFAULT_TABLE table=test-table [id=dadd84abc95f483893aa710fb7282f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:31.257663 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57aeefc6797e44389645dcd3b8026ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:31.259756 18652 tablet_bootstrap.cc:492] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Bootstrap starting.
I20260926 13:50:31.260106 18653 tablet_bootstrap.cc:492] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38: Bootstrap starting.
I20260926 13:50:31.261036 18652 tablet_bootstrap.cc:654] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:31.261186 18653 tablet_bootstrap.cc:654] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:31.261822 18654 tablet_bootstrap.cc:492] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Bootstrap starting.
I20260926 13:50:31.262331 18652 tablet_bootstrap.cc:492] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: No bootstrap required, opened a new log
I20260926 13:50:31.262420 18653 tablet_bootstrap.cc:492] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38: No bootstrap required, opened a new log
I20260926 13:50:31.262411 18652 ts_tablet_manager.cc:1397] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:31.262516 18653 ts_tablet_manager.cc:1397] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:31.262871 18653 raft_consensus.cc:348] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.262876 18652 raft_consensus.cc:348] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.263005 18652 raft_consensus.cc:374] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:31.263008 18653 raft_consensus.cc:374] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:31.263027 18652 raft_consensus.cc:729] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5e28e1cedb4b0c9f47d46deffa2ff7, State: Initialized, Role: FOLLOWER
I20260926 13:50:31.263058 18653 raft_consensus.cc:729] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c08d8b4d0874f5e948507bb37f8ff38, State: Initialized, Role: FOLLOWER
I20260926 13:50:31.263192 18653 consensus_queue.cc:260] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.263190 18652 consensus_queue.cc:260] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.263623 18653 ts_tablet_manager.cc:1428] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:31.263621 18652 ts_tablet_manager.cc:1428] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 13:50:31.264246 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.263923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:47487)
0926 13:50:31.264106 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.264168 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 13:50:31.264297 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.263923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:49041)
0926 13:50:31.264150 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.264221 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260926 13:50:31.264755 18654 tablet_bootstrap.cc:654] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:31.266189 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.265951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:58263)
0926 13:50:31.266067 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.266119 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 13:50:31.266219 18654 tablet_bootstrap.cc:492] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: No bootstrap required, opened a new log
I20260926 13:50:31.266351 18654 ts_tablet_manager.cc:1397] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 13:50:31.266930 18654 raft_consensus.cc:348] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.267053 18654 raft_consensus.cc:374] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:31.267088 18654 raft_consensus.cc:729] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f429a955862402981ded49a581ba755, State: Initialized, Role: FOLLOWER
I20260926 13:50:31.267242 18654 consensus_queue.cc:260] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.267424 18278 heartbeater.cc:503] Master 127.10.236.252:34915 was elected leader, sending a full tablet report...
I20260926 13:50:31.267715 18654 ts_tablet_manager.cc:1428] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 13:50:31.268393 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.268017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:50447)
0926 13:50:31.268251 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.268318 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260926 13:50:31.269349 18415 heartbeater.cc:503] Master 127.10.236.252:34915 was elected leader, sending a full tablet report...
I20260926 13:50:31.269564 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.269129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:47031)
0926 13:50:31.269284 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.269336 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 13:50:31.270059 18552 heartbeater.cc:503] Master 127.10.236.252:34915 was elected leader, sending a full tablet report...
I20260926 13:50:31.270430 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.270201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:55747)
0926 13:50:31.270311 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.270363 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:50:31.297361 18554 tablet.cc:2369] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:50:31.348989 18417 tablet.cc:2369] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:31.589500 18658 raft_consensus.cc:482] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:31.589664 18658 raft_consensus.cc:504] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.590899 18658 leader_election.cc:288] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f429a955862402981ded49a581ba755 (127.10.236.195:42945), 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001)
I20260926 13:50:31.596450 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.591250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:51251 (local address 127.10.236.193:36001)
0926 13:50:31.591359 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:31.591365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:31.591553 (+ 188us) server_negotiation.cc:407] Connection header received
0926 13:50:31.591653 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:31.591656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:31.591726 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:31.591840 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:31.592198 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.593112 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.595155 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.595801 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.595954 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.596012 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:31.596252 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 13:50:31.596327 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:50:31.596472 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.591161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36001 (local address 127.10.236.194:51251)
0926 13:50:31.591358 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.591374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:31.591524 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:31.591867 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:31.591869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:31.591884 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:31.592034 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.592041 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.593264 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.593268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.595008 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.595014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.595974 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.595979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.596086 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.596115 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 13:50:31.596215 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 13:50:31.596305 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":239}
I20260926 13:50:31.597062 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.591285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:52825 (local address 127.10.236.195:42945)
0926 13:50:31.591504 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:31.591509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:31.591660 (+ 151us) server_negotiation.cc:407] Connection header received
0926 13:50:31.591801 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:31.591808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:31.591864 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:31.591944 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:31.592210 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.593112 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.595610 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.596248 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.596434 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.596505 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:31.596868 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 13:50:31.596943 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:50:31.597086 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.591193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:42945 (local address 127.10.236.194:52825)
0926 13:50:31.591560 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.591574 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:31.591691 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:31.591986 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:31.591992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:31.592010 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:31.592081 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.592089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.593264 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.593268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.595002 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:31.595014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.596574 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.596579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:31.596703 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.596734 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 13:50:31.596840 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:50:31.596919 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":184,"thread_start_us":152,"threads_started":1}
I20260926 13:50:31.597090 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57aeefc6797e44389645dcd3b8026ab2" candidate_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" is_pre_election: true
I20260926 13:50:31.597317 18231 raft_consensus.cc:2454] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5e28e1cedb4b0c9f47d46deffa2ff7 in term 0.
I20260926 13:50:31.597751 18302 leader_election.cc:302] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5e28e1cedb4b0c9f47d46deffa2ff7, 9c08d8b4d0874f5e948507bb37f8ff38; no voters:
I20260926 13:50:31.598026 18658 raft_consensus.cc:2781] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:31.598116 18658 raft_consensus.cc:482] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:31.598176 18658 raft_consensus.cc:3037] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:31.598220 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57aeefc6797e44389645dcd3b8026ab2" candidate_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f429a955862402981ded49a581ba755" is_pre_election: true
I20260926 13:50:31.598364 18505 raft_consensus.cc:2454] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5e28e1cedb4b0c9f47d46deffa2ff7 in term 0.
I20260926 13:50:31.599685 18658 raft_consensus.cc:504] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.600118 18658 leader_election.cc:288] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [CANDIDATE]: Term 1 election: Requested vote from peers 9f429a955862402981ded49a581ba755 (127.10.236.195:42945), 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001)
I20260926 13:50:31.600390 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57aeefc6797e44389645dcd3b8026ab2" candidate_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f429a955862402981ded49a581ba755"
I20260926 13:50:31.600510 18505 raft_consensus.cc:3037] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:31.601539 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57aeefc6797e44389645dcd3b8026ab2" candidate_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c08d8b4d0874f5e948507bb37f8ff38"
I20260926 13:50:31.601636 18505 raft_consensus.cc:2454] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5e28e1cedb4b0c9f47d46deffa2ff7 in term 1.
I20260926 13:50:31.601681 18231 raft_consensus.cc:3037] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:31.601908 18303 leader_election.cc:302] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5e28e1cedb4b0c9f47d46deffa2ff7, 9f429a955862402981ded49a581ba755; no voters:
I20260926 13:50:31.602113 18658 raft_consensus.cc:2781] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:31.602370 18658 raft_consensus.cc:686] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5e28e1cedb4b0c9f47d46deffa2ff7, State: Running, Role: LEADER
I20260926 13:50:31.602640 18658 consensus_queue.cc:237] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } }
I20260926 13:50:31.602823 18231 raft_consensus.cc:2454] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5e28e1cedb4b0c9f47d46deffa2ff7 in term 1.
I20260926 13:50:31.603649 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.603305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:34187)
0926 13:50:31.603453 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.603586 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 13:50:31.605211 18097 catalog_manager.cc:5495] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5e28e1cedb4b0c9f47d46deffa2ff7" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:31.623342 18647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.619736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.10.236.194:36329)
0926 13:50:31.619868 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:31.619871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:31.619883 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:50:31.619969 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:31.619971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:31.620021 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:31.620108 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:31.620420 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.620906 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.621907 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:31.622128 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:31.622212 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:31.622514 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:31.622975 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:31.623062 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:31.623181 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 13:50:31.623232 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:50:31.626368 18505 raft_consensus.cc:1261] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Refusing update from remote peer 8b5e28e1cedb4b0c9f47d46deffa2ff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:31.626597 18231 raft_consensus.cc:1261] T 57aeefc6797e44389645dcd3b8026ab2 P 9c08d8b4d0874f5e948507bb37f8ff38 [term 1 FOLLOWER]: Refusing update from remote peer 8b5e28e1cedb4b0c9f47d46deffa2ff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:31.626963 18658 consensus_queue.cc:1035] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f429a955862402981ded49a581ba755" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:31.627152 18662 consensus_queue.cc:1035] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c08d8b4d0874f5e948507bb37f8ff38" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:31.629242 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.628747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:58135)
0926 13:50:31.629087 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.629158 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
I20260926 13:50:31.629475 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.629188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:38213)
0926 13:50:31.629341 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.629399 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 13:50:31.632910 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.632542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:59639)
0926 13:50:31.632759 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.632824 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260926 13:50:31.633683 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.633424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.194:45275)
0926 13:50:31.633566 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.633615 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 13:50:31.634091 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.633075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.195:37053)
0926 13:50:31.633960 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.634016 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":808}
I20260926 13:50:31.634560 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.633236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36389 (local address 127.10.236.193:34745)
0926 13:50:31.634433 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.634488 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1124}
I20260926 13:50:31.642541 18095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:31.642685 18095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:31.646759 18095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cfbadee761a4efd81b106fd90c1e717: Sending DeleteTablet for 3 replicas of tablet 57aeefc6797e44389645dcd3b8026ab2
I20260926 13:50:31.647696 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet 57aeefc6797e44389645dcd3b8026ab2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:31 UTC) from {username='oryx', principal='oryx/127.10.236.252@KRBTEST.COM'} at 127.0.0.1:54126
I20260926 13:50:31.647688 18348 tablet_service.cc:1514] Processing DeleteTablet for tablet 57aeefc6797e44389645dcd3b8026ab2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:31 UTC) from {username='oryx', principal='oryx/127.10.236.252@KRBTEST.COM'} at 127.0.0.1:40212
I20260926 13:50:31.648096 18485 tablet_service.cc:1514] Processing DeleteTablet for tablet 57aeefc6797e44389645dcd3b8026ab2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:31 UTC) from {username='oryx', principal='oryx/127.10.236.252@KRBTEST.COM'} at 127.0.0.1:59986
I20260926 13:50:31.648454 18677 tablet_replica.cc:330] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: stopping tablet replica
I20260926 13:50:31.648461 18675 tablet_replica.cc:330] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: stopping tablet replica
I20260926 13:50:31.648595 18675 raft_consensus.cc:2229] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:31.648715 18677 raft_consensus.cc:2229] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:31.648864 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 18059
I20260926 13:50:31.648936 18675 raft_consensus.cc:2258] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:31.650223 18675 ts_tablet_manager.cc:1905] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:31.651777 18677 raft_consensus.cc:2258] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:31.652820 18677 ts_tablet_manager.cc:1905] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:31.653900 18675 ts_tablet_manager.cc:1918] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:31.653990 18675 log.cc:1196] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/57aeefc6797e44389645dcd3b8026ab2
I20260926 13:50:31.654277 18675 ts_tablet_manager.cc:1939] T 57aeefc6797e44389645dcd3b8026ab2 P 8b5e28e1cedb4b0c9f47d46deffa2ff7: Deleting consensus metadata
I20260926 13:50:31.655669 18083 catalog_manager.cc:4841] TS 8b5e28e1cedb4b0c9f47d46deffa2ff7 (127.10.236.194:36329): tablet 57aeefc6797e44389645dcd3b8026ab2 (table test-table [id=dadd84abc95f483893aa710fb7282f80]) successfully deleted
I20260926 13:50:31.657014 18677 ts_tablet_manager.cc:1918] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:31.657162 18677 log.cc:1196] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/57aeefc6797e44389645dcd3b8026ab2
I20260926 13:50:31.657501 18677 ts_tablet_manager.cc:1939] T 57aeefc6797e44389645dcd3b8026ab2 P 9f429a955862402981ded49a581ba755: Deleting consensus metadata
I20260926 13:50:31.657944 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 18285
I20260926 13:50:31.658699 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:31.658030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36001 (local address 127.0.0.1:54136)
0926 13:50:31.658533 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:31.658594 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:36001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401}
W20260926 13:50:31.659497 18083 catalog_manager.cc:4584] TS 9c08d8b4d0874f5e948507bb37f8ff38 (127.10.236.193:36001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57aeefc6797e44389645dcd3b8026ab2: Network error: Client connection negotiation failed: client connection to 127.10.236.193:36001: connect: Connection refused (error 111)
I20260926 13:50:31.659600 18084 catalog_manager.cc:4841] TS 9f429a955862402981ded49a581ba755 (127.10.236.195:42945): tablet 57aeefc6797e44389645dcd3b8026ab2 (table test-table [id=dadd84abc95f483893aa710fb7282f80]) successfully deleted
I20260926 13:50:31.664222 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 18421
I20260926 13:50:31.670472 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17776
I20260926 13:50:31.677137 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 17880
2026-09-26T13:50:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7300 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:31 dist-test-slave-r34j krb5kdc[18680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:31 dist-test-slave-r34j krb5kdc[18680](info): set up 2 sockets
Sep 26 13:50:31 dist-test-slave-r34j krb5kdc[18680](info): commencing operation
krb5kdc: starting...
W20260926 13:50:32.736043 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:32 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab.
Entry for principal oryx/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab.
I20260926 13:50:32.863371 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33855
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab
--principal=oryx/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:33855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:32.976450 18693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:32.976763 18693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:32.976804 18693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:32.981091 18693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:32.981164 18693 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 13:50:32.981184 18693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:32.981201 18693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:32.981218 18693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:32.981235 18693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:32.986972 18693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:33855
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.254.keytab
--principal=oryx/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:32.988196 18693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:32.989275 18693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:32.998080 18698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:32.998106 18701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:32.998072 18699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:32.998343 18693 server_base.cc:1034] running on GCE node
I20260926 13:50:32.999042 18693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:33.000232 18693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:33.001385 18693 hybrid_clock.cc:648] HybridClock initialized: now 1790430633001356 us; error 51 us; skew 500 ppm
Sep 26 13:50:32 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430632, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:33.004141 18693 init.cc:377] Logged in from keytab as oryx/127.10.236.254@KRBTEST.COM (short username oryx)
I20260926 13:50:33.005091 18693 webserver.cc:460] Webserver started at http://127.10.236.254:42857/ using document root <none> and password file <none>
I20260926 13:50:33.005662 18693 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:33.005757 18693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:33.005996 18693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:33.008127 18693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "e35cad7b7ffa471e81fbf9f32517e4f4"
format_stamp: "Formatted at 2026-09-26 13:50:32 on dist-test-slave-r34j"
server_key: "b9d9a8a82e1c542861987d78cc61aa46"
server_key_iv: "fac5d8fc52fb9b029de7395a54355eaf"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.008693 18693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "e35cad7b7ffa471e81fbf9f32517e4f4"
format_stamp: "Formatted at 2026-09-26 13:50:32 on dist-test-slave-r34j"
server_key: "b9d9a8a82e1c542861987d78cc61aa46"
server_key_iv: "fac5d8fc52fb9b029de7395a54355eaf"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.012491 18693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:33.015019 18708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.016546 18693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:33.016702 18693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "e35cad7b7ffa471e81fbf9f32517e4f4"
format_stamp: "Formatted at 2026-09-26 13:50:32 on dist-test-slave-r34j"
server_key: "b9d9a8a82e1c542861987d78cc61aa46"
server_key_iv: "fac5d8fc52fb9b029de7395a54355eaf"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.016829 18693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:33.028725 18693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:33.031988 18693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:33.032207 18693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:33.040411 18693 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:33855
I20260926 13:50:33.040439 18770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:33855 every 8 connection(s)
I20260926 13:50:33.041428 18693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:33.044246 18771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.049631 18771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Bootstrap starting.
I20260926 13:50:33.050007 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 18693
I20260926 13:50:33.050120 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:33.050411 11187 external_mini_cluster.cc:1428] Setting key 93f3828204367e024bb25752e64b806c
I20260926 13:50:33.052454 18771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.053176 18771 log.cc:824] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:33.055373 18771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: No bootstrap required, opened a new log
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:33.059948 18771 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33855 } }
I20260926 13:50:33.060293 18771 raft_consensus.cc:374] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.060359 18771 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35cad7b7ffa471e81fbf9f32517e4f4, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.060808 18771 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33855 } }
I20260926 13:50:33.060959 18771 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:33.061044 18771 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:33.061174 18771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:33.062379 18771 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33855 } }
I20260926 13:50:33.062786 18771 leader_election.cc:302] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35cad7b7ffa471e81fbf9f32517e4f4; no voters:
I20260926 13:50:33.063159 18771 leader_election.cc:288] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:33.063285 18776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:33.063524 18776 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [term 1 LEADER]: Becoming Leader. State: Replica: e35cad7b7ffa471e81fbf9f32517e4f4, State: Running, Role: LEADER
I20260926 13:50:33.063879 18776 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33855 } }
I20260926 13:50:33.064571 18771 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:33.065100 18776 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35cad7b7ffa471e81fbf9f32517e4f4. Latest consensus state: current_term: 1 leader_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33855 } } }
I20260926 13:50:33.065215 18776 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:33.066161 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.051816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.10.236.254:33855)
0926 13:50:33.052288 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.052308 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.052345 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:50:33.053061 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.053079 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.053461 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.053850 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.054697 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.055823 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.057003 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.057487 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.057618 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.060549 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.060554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.060568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.060618 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.062660 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.063140 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.063146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.063153 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.063313 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.063751 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.063754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.063755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.064274 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.064461 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.064786 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.065091 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":140,"threads_started":1}
I20260926 13:50:33.066936 18778 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35cad7b7ffa471e81fbf9f32517e4f4" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33855 } } }
I20260926 13:50:33.067024 18778 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4 [sys.catalog]: This master's current role is: LEADER
W20260926 13:50:33.070058 18791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:50:33.070122 18791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:50:33.070204 18782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:33.070878 18782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:33.076910 18782 catalog_manager.cc:1347] Generated new cluster ID: 10b0fcd192db4e5eb5b66278c2f5b7ed
I20260926 13:50:33.076992 18782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:33.085538 18782 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:33.087267 18782 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:33.103621 18782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Generated new TSK 0
I20260926 13:50:33.104553 18782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab.
Entry for principal oryx/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab.
I20260926 13:50:33.138978 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33855
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:33.252627 18797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:33.252908 18797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:33.252954 18797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:33.257125 18797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:33.257189 18797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:33.257229 18797 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:33.257297 18797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:33.262826 18797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:33855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:33.264124 18797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:33.265285 18797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:33.275683 18797 server_base.cc:1034] running on GCE node
W20260926 13:50:33.275630 18802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:33.275630 18803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:33.275629 18805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:33.276444 18797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:33.277120 18797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:33.278277 18797 hybrid_clock.cc:648] HybridClock initialized: now 1790430633278256 us; error 36 us; skew 500 ppm
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:33.281145 18797 init.cc:377] Logged in from keytab as oryx/127.10.236.193@KRBTEST.COM (short username oryx)
I20260926 13:50:33.282078 18797 webserver.cc:460] Webserver started at http://127.10.236.193:45707/ using document root <none> and password file <none>
I20260926 13:50:33.282646 18797 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:33.282737 18797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:33.282974 18797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:33.285113 18797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "55b4ccb49b167c2c998bd651db93c3f4"
server_key_iv: "f315f1bbb6c0bb04fb8ee7c47d1782ef"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.285701 18797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "55b4ccb49b167c2c998bd651db93c3f4"
server_key_iv: "f315f1bbb6c0bb04fb8ee7c47d1782ef"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.289984 18797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260926 13:50:33.292702 18812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.294052 18797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:33.294203 18797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "55b4ccb49b167c2c998bd651db93c3f4"
server_key_iv: "f315f1bbb6c0bb04fb8ee7c47d1782ef"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.294328 18797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:33.306350 18797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:33.309418 18797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:33.309629 18797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:33.311054 18797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:33.311133 18797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.311209 18797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:33.311254 18797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:33.322281 18797 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:42347
I20260926 13:50:33.322340 18927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:42347 every 8 connection(s)
I20260926 13:50:33.323200 18797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:33.325708 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.312031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44699 (local address 127.10.236.254:33855)
0926 13:50:33.312154 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.312158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.313253 (+ 1095us) server_negotiation.cc:407] Connection header received
0926 13:50:33.314304 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.314307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.314368 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.314480 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.316282 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.317174 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.318605 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.319022 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.319126 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.322133 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.322138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.322142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.322171 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.323863 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.324476 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.324479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.324480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.324557 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.324820 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.324822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.324823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.324992 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.325066 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.325498 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.325594 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:50:33.326532 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.312586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.10.236.193:44699)
0926 13:50:33.313102 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.313143 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.314085 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.314711 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.314721 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.315245 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:33.316077 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.316102 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.317305 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.317310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.318461 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.318470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.319198 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.319202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.319335 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.320007 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:33.320031 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:33.321900 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:33.324044 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.324050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.324063 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.324359 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.324630 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.324632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.324633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.324746 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.325078 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:33.325083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:33.325293 (+ 210us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.325442 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.325670 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":144,"threads_started":1}
I20260926 13:50:33.326779 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 18797
I20260926 13:50:33.326926 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:33.327288 11187 external_mini_cluster.cc:1428] Setting key 7f9ee69eb13c5606b3a1fc7bf1b9e9de
I20260926 13:50:33.327922 18928 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33855
I20260926 13:50:33.328052 18928 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:33.328578 18928 heartbeater.cc:511] Master 127.10.236.254:33855 requested a full tablet report, sending...
I20260926 13:50:33.330225 18723 ts_manager.cc:194] Registered new tserver with Master: 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347)
I20260926 13:50:33.331596 18723 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:44699
WARNING: no policy specified for oryx/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:50:33.338384 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.330109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:40203 (local address 127.10.236.254:33855)
0926 13:50:33.330266 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.330270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.330686 (+ 416us) server_negotiation.cc:407] Connection header received
0926 13:50:33.330861 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.330864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.330907 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.330988 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.331537 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.332232 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.333397 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.333693 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.333764 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.335032 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.335036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.335039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.335061 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.336674 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.337119 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.337124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.337126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.337207 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.337496 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.337500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.337502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.337692 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.337774 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.338093 (+ 319us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.338185 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:50:33.338302 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.330229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.10.236.193:40203)
0926 13:50:33.330597 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.330613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.330754 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.331022 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.331026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.331255 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:33.331398 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.331407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.332335 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.332337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.333266 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.333276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.333924 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.333927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.334026 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.334549 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:33.334569 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:33.334885 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:33.336807 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.336811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.336813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.337021 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.337317 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.337319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.337321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.337390 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.337858 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:33.337860 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:33.337943 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.338040 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.338150 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab.
Entry for principal oryx/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab.
I20260926 13:50:33.355594 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33855
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:33.468677 18937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:33.468999 18937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:33.469087 18937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:33.473670 18937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:33.473834 18937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:33.473932 18937 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:33.474006 18937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:33.479532 18937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:33855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:33.480634 18937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:33.481693 18937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:33.492004 18942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:33.491915 18943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:33.492041 18937 server_base.cc:1034] running on GCE node
W20260926 13:50:33.492241 18945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:33.492686 18937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:33.493355 18937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:33.494575 18937 hybrid_clock.cc:648] HybridClock initialized: now 1790430633494532 us; error 53 us; skew 500 ppm
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:33.497674 18937 init.cc:377] Logged in from keytab as oryx/127.10.236.194@KRBTEST.COM (short username oryx)
I20260926 13:50:33.498643 18937 webserver.cc:460] Webserver started at http://127.10.236.194:43713/ using document root <none> and password file <none>
I20260926 13:50:33.499182 18937 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:33.499238 18937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:33.499403 18937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:33.501734 18937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "158a311c4c6d4fd0a2b01920fc17df58"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "e9255ca92b20aacc3c5b7979eb761e14"
server_key_iv: "6e04e4943a1af57f44a2c01f9052a44a"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.502295 18937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "158a311c4c6d4fd0a2b01920fc17df58"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "e9255ca92b20aacc3c5b7979eb761e14"
server_key_iv: "6e04e4943a1af57f44a2c01f9052a44a"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.506184 18937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:33.508847 18952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.510346 18937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:50:33.510500 18937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "158a311c4c6d4fd0a2b01920fc17df58"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "e9255ca92b20aacc3c5b7979eb761e14"
server_key_iv: "6e04e4943a1af57f44a2c01f9052a44a"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.510625 18937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:33.526965 18937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:33.530723 18937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:33.530941 18937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:33.532572 18937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:33.532645 18937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.532716 18937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:33.532768 18937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:33.543807 18937 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33189
I20260926 13:50:33.543865 19067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33189 every 8 connection(s)
I20260926 13:50:33.544735 18937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:33.546247 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.533400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:50877 (local address 127.10.236.254:33855)
0926 13:50:33.533572 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.533579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.534372 (+ 793us) server_negotiation.cc:407] Connection header received
0926 13:50:33.535653 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.535657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.535720 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.535776 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.537118 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.537665 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.538909 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.539194 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.539311 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.542446 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.542450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.542453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.542487 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.544141 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.544644 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.544649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.544651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.544736 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.545077 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.545080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.545082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.545328 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.545421 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.546021 (+ 600us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.546128 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:33.547027 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.533783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.10.236.194:50877)
0926 13:50:33.534230 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.534279 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.535333 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.536185 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.536192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.536512 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:33.536978 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.536990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.537795 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.537798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.538785 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.538794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.539361 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.539365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.539599 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.540306 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:33.540326 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:33.542275 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:33.544253 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.544259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.544274 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.544540 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.544829 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.544832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.544834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.544973 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.545472 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:33.545480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:33.545767 (+ 287us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.545947 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.546196 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":133,"threads_started":1}
I20260926 13:50:33.548347 19068 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33855
I20260926 13:50:33.548523 19068 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:33.549010 19068 heartbeater.cc:511] Master 127.10.236.254:33855 requested a full tablet report, sending...
I20260926 13:50:33.550032 18724 ts_manager.cc:194] Registered new tserver with Master: 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
I20260926 13:50:33.550840 18724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:50877
I20260926 13:50:33.552340 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 18937
I20260926 13:50:33.552618 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:33.552980 11187 external_mini_cluster.cc:1428] Setting key c30f7683010a80e616715353c15c343e
I20260926 13:50:33.558104 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.550525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:53859 (local address 127.10.236.254:33855)
0926 13:50:33.550652 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.550656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.550820 (+ 164us) server_negotiation.cc:407] Connection header received
0926 13:50:33.550993 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.550996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.551034 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.551122 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.551689 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.552406 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.553558 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.553792 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.553872 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.555068 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.555071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.555074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.555100 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.556463 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.556874 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.556879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.556881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.556962 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.557230 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.557235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.557236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.557444 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.557581 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.557838 (+ 257us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.557944 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:33.558118 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.550464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.10.236.194:53859)
0926 13:50:33.550748 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.550764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.550877 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.551170 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.551173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.551395 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:33.551584 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.551591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.552672 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.552677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.553466 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.553475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.553935 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.553939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.554037 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.554619 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:33.554630 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:33.554938 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:33.556571 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.556577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.556580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.556778 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.557049 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.557053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.557055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.557142 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.557569 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:33.557572 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:33.557673 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.557777 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.557932 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.10.236.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab.
Entry for principal oryx/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab.
I20260926 13:50:33.583209 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33855
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:33.697763 19077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:33.698136 19077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:33.698227 19077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:33.702504 19077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:33.702613 19077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:33.702692 19077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:50:33.702771 19077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:33.708263 19077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.236.254:33855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:33.709396 19077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:33.710441 19077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:33.720464 19083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:33.720455 19082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:33.720455 19085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:33.720809 19077 server_base.cc:1034] running on GCE node
I20260926 13:50:33.721355 19077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:33.722210 19077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:33.723392 19077 hybrid_clock.cc:648] HybridClock initialized: now 1790430633723357 us; error 54 us; skew 500 ppm
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:33.726655 19077 init.cc:377] Logged in from keytab as oryx/127.10.236.195@KRBTEST.COM (short username oryx)
I20260926 13:50:33.727795 19077 webserver.cc:460] Webserver started at http://127.10.236.195:34905/ using document root <none> and password file <none>
I20260926 13:50:33.728490 19077 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:33.728559 19077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:33.728785 19077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:33.731127 19077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "ef4305690b9240c784aa1b654e376ddf"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "a9bbebe70a9aeb48ea478b437483a3bc"
server_key_iv: "4d1db9f14263c3038a99a6a73da8bfe1"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.731781 19077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "ef4305690b9240c784aa1b654e376ddf"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "a9bbebe70a9aeb48ea478b437483a3bc"
server_key_iv: "4d1db9f14263c3038a99a6a73da8bfe1"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.735947 19077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:50:33.738690 19092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.740185 19077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:33.740288 19077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "ef4305690b9240c784aa1b654e376ddf"
format_stamp: "Formatted at 2026-09-26 13:50:33 on dist-test-slave-r34j"
server_key: "a9bbebe70a9aeb48ea478b437483a3bc"
server_key_iv: "4d1db9f14263c3038a99a6a73da8bfe1"
server_key_version: "encryptionkey@0"
I20260926 13:50:33.740435 19077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:33.750015 19077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:33.753538 19077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:33.753744 19077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:33.755254 19077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:33.755329 19077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.755398 19077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:33.755451 19077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:33.766759 19077 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:35943
I20260926 13:50:33.766892 19207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:35943 every 8 connection(s)
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:33.768209 19077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:33.770017 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19077
I20260926 13:50:33.770175 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:33.770445 11187 external_mini_cluster.cc:1428] Setting key 8391c1cd20b0c162c06da1695ea98996
I20260926 13:50:33.772773 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.756429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:49373 (local address 127.10.236.254:33855)
0926 13:50:33.756566 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.756570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.757708 (+ 1138us) server_negotiation.cc:407] Connection header received
0926 13:50:33.759252 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.759257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.759329 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.759425 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.761221 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.762124 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.763854 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.764246 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.764383 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.768146 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.768149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.768151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.768179 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.770019 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.770899 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.770903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.770905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.771054 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.771529 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.771532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.771534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.771745 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.771860 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.772484 (+ 624us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.772614 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:33.773614 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.756993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.10.236.195:49373)
0926 13:50:33.757539 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.757584 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.758947 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.759677 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.759691 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.760231 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:33.760994 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.761013 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.762291 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.762295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.763706 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.763715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.764530 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.764541 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.764899 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.765681 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:33.765700 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:33.767915 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:33.770368 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.770375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.770399 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.770755 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.771169 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.771172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.771173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.771377 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.771986 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:33.771992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:33.772236 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.772427 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.772780 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":142,"threads_started":1}
I20260926 13:50:33.774899 19208 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33855
I20260926 13:50:33.775069 19208 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:33.775650 19208 heartbeater.cc:511] Master 127.10.236.254:33855 requested a full tablet report, sending...
I20260926 13:50:33.776693 18724 ts_manager.cc:194] Registered new tserver with Master: ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943)
I20260926 13:50:33.777484 18724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:49373
I20260926 13:50:33.785410 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.777485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:57771 (local address 127.10.236.254:33855)
0926 13:50:33.777642 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.777646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.777962 (+ 316us) server_negotiation.cc:407] Connection header received
0926 13:50:33.778099 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.778102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.778150 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.778234 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.778700 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.779456 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.780284 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.780517 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.780602 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.781748 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.781751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.781753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.781774 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.783553 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.784114 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.784117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.784118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.784195 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.784500 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.784503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.784505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.784748 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.784865 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.785141 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.785249 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:50:33.785434 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.777357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.10.236.195:57771)
0926 13:50:33.777869 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.777885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.777989 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.778255 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.778258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.778487 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:33.778587 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.778592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.779604 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.779607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.780167 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.780175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.780629 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.780631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.780708 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.781196 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:33.781207 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:33.781622 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:33.783712 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.783715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.783716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.784013 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.784296 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:33.784299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:33.784300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:33.784390 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:33.784853 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:33.784856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:33.784966 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.785076 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.785236 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":141,"threads_started":1}
I20260926 13:50:33.786409 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:50:33.811425 18774 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:33.804330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.10.236.254:33855)
0926 13:50:33.804499 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.804503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.804525 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:50:33.804566 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.804569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.804609 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.804681 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.805534 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.806103 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.807101 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.807337 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.807426 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.811169 (+ 3743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57326: BlockingRecv error: recv got EOF from 127.0.0.1:57326 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 26 13:50:33 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:33.821475 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.812970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.10.236.254:33855)
0926 13:50:33.813120 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.813124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.813253 (+ 129us) server_negotiation.cc:407] Connection header received
0926 13:50:33.813403 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.813406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.813486 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.813575 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:33.814226 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.814737 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.815640 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.815858 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.815965 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.818352 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:33.818355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:33.818357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:33.818382 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:33.819743 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.820231 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.820239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.820241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.820317 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:33.820621 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:33.820626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:33.820628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:33.820811 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:33.820995 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.821192 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.821310 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:50:33.824662 18724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:33.827749 18724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:33.842401 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.836562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.10.236.194:33189)
0926 13:50:33.837307 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.837320 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.837342 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:50:33.837431 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.837437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.837653 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.837853 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:33.838158 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.839365 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.841030 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.841745 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.841914 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.842050 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.842153 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.842240 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":89,"threads_started":1}
I20260926 13:50:33.842397 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.836256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33189 (local address 127.0.0.1:51196)
0926 13:50:33.836734 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.836781 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.836921 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.837863 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.837869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.837891 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:33.838035 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.838043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.839477 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.839505 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.840894 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.840903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.841869 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.841873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.841973 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.842008 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.842127 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.842250 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":115,"spinlock_wait_cycles":3328,"thread_start_us":103,"threads_started":1}
I20260926 13:50:33.842599 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.836474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35943 (local address 127.0.0.1:35976)
0926 13:50:33.836939 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.836953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.837042 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.838046 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.838049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.838062 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:33.838165 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.838171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.839369 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.839372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.841016 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.841023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.842146 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.842149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.842224 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.842277 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.842344 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.842436 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":76,"thread_start_us":100,"threads_started":1}
I20260926 13:50:33.842880 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.837384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.10.236.193:42347)
0926 13:50:33.837724 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.837733 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.838010 (+ 277us) server_negotiation.cc:407] Connection header received
0926 13:50:33.838087 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.838093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.838293 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.838428 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:33.838633 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.839675 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.840972 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.841792 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.841912 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.842051 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.842432 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.842743 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":91,"threads_started":1}
I20260926 13:50:33.843196 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.837001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:42347 (local address 127.0.0.1:39218)
0926 13:50:33.837833 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.837845 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.837943 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.838419 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.838422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.838439 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:33.838511 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.838517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.839789 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.839795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.840856 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.840864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.842115 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.842121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.842220 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.842262 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.842345 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.843070 (+ 725us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":662,"spinlock_wait_cycles":2560,"thread_start_us":84,"threads_started":1}
I20260926 13:50:33.843658 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.837058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.10.236.195:35943)
0926 13:50:33.837587 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.837601 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.837624 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:50:33.837699 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.837704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.837919 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.838057 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:33.838286 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.839246 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.841120 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.841883 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.842004 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.842101 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.842838 (+ 737us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.842942 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":113,"threads_started":1}
I20260926 13:50:33.845760 19002 tablet_service.cc:1467] Processing CreateTablet for tablet c6224402742f471286f73bc080ce9780 (DEFAULT_TABLE table=test-table [id=1a863e95f58342b08b006097042426c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:33.845748 18862 tablet_service.cc:1467] Processing CreateTablet for tablet c6224402742f471286f73bc080ce9780 (DEFAULT_TABLE table=test-table [id=1a863e95f58342b08b006097042426c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:33.846906 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6224402742f471286f73bc080ce9780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.846917 18862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6224402742f471286f73bc080ce9780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.849874 19142 tablet_service.cc:1467] Processing CreateTablet for tablet c6224402742f471286f73bc080ce9780 (DEFAULT_TABLE table=test-table [id=1a863e95f58342b08b006097042426c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:33.850987 19142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6224402742f471286f73bc080ce9780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.854684 19236 tablet_bootstrap.cc:492] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Bootstrap starting.
I20260926 13:50:33.856068 19237 tablet_bootstrap.cc:492] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Bootstrap starting.
I20260926 13:50:33.857496 19236 tablet_bootstrap.cc:654] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.857662 19238 tablet_bootstrap.cc:492] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Bootstrap starting.
I20260926 13:50:33.857882 19237 tablet_bootstrap.cc:654] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.858357 19236 log.cc:824] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:33.858680 19237 log.cc:824] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:33.859747 19238 tablet_bootstrap.cc:654] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.860189 19236 tablet_bootstrap.cc:492] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: No bootstrap required, opened a new log
I20260926 13:50:33.860386 19236 ts_tablet_manager.cc:1397] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:50:33.860423 19238 log.cc:824] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:33.862254 19238 tablet_bootstrap.cc:492] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: No bootstrap required, opened a new log
I20260926 13:50:33.862442 19238 ts_tablet_manager.cc:1397] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:50:33.864230 19236 raft_consensus.cc:348] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.864467 19236 raft_consensus.cc:374] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.864549 19236 raft_consensus.cc:729] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f934ba15ebb4ae2a85c023dbff446b5, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.864992 19236 consensus_queue.cc:260] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.865638 18928 heartbeater.cc:503] Master 127.10.236.254:33855 was elected leader, sending a full tablet report...
I20260926 13:50:33.865932 19236 ts_tablet_manager.cc:1428] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260926 13:50:33.865808 19238 raft_consensus.cc:348] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.866032 19238 raft_consensus.cc:374] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.866094 19238 raft_consensus.cc:729] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.866508 19238 consensus_queue.cc:260] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.867173 19208 heartbeater.cc:503] Master 127.10.236.254:33855 was elected leader, sending a full tablet report...
I20260926 13:50:33.867518 19238 ts_tablet_manager.cc:1428] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 13:50:33.868048 19237 tablet_bootstrap.cc:492] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: No bootstrap required, opened a new log
I20260926 13:50:33.868249 19237 ts_tablet_manager.cc:1397] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260926 13:50:33.871789 19237 raft_consensus.cc:348] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.872010 19237 raft_consensus.cc:374] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.872043 19237 raft_consensus.cc:729] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158a311c4c6d4fd0a2b01920fc17df58, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.872470 19237 consensus_queue.cc:260] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.873075 19068 heartbeater.cc:503] Master 127.10.236.254:33855 was elected leader, sending a full tablet report...
I20260926 13:50:33.873317 19237 ts_tablet_manager.cc:1428] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 13:50:33.912967 19244 raft_consensus.cc:482] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:33.913164 19244 raft_consensus.cc:504] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.914484 19244 leader_election.cc:288] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943), 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347)
I20260926 13:50:33.918991 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.914945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:51717 (local address 127.10.236.195:35943)
0926 13:50:33.915112 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.915118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.915138 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:50:33.915197 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.915203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.915253 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.915337 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:33.915619 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.916180 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.917976 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.918374 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.918506 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.918567 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.918786 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.918854 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 13:50:33.918993 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.915196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:33557 (local address 127.10.236.193:42347)
0926 13:50:33.915328 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.915333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.915573 (+ 240us) server_negotiation.cc:407] Connection header received
0926 13:50:33.915690 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.915693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.915742 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.915823 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:33.916115 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.916894 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.917967 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.918376 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.918506 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.918567 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.918779 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.918855 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 13:50:33.919014 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.915067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:42347 (local address 127.10.236.194:33557)
0926 13:50:33.915453 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.915468 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.915606 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.915842 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.915845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.915860 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:33.915985 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.915990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.917028 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.917031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.917842 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.917851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.918499 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.918503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.918654 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.918672 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.918763 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.918848 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":28,"thread_start_us":122,"threads_started":1}
I20260926 13:50:33.919009 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.914751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35943 (local address 127.10.236.194:51717)
0926 13:50:33.914905 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:33.914921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:33.915049 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:33.915345 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:33.915348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:33.915364 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:33.915511 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.915517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.916300 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.916303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.917838 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:33.917847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.918498 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.918502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:33.918649 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.918675 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:50:33.918760 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 13:50:33.918863 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":25}
I20260926 13:50:33.919691 19162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6224402742f471286f73bc080ce9780" candidate_uuid: "158a311c4c6d4fd0a2b01920fc17df58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef4305690b9240c784aa1b654e376ddf" is_pre_election: true
I20260926 13:50:33.919669 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6224402742f471286f73bc080ce9780" candidate_uuid: "158a311c4c6d4fd0a2b01920fc17df58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" is_pre_election: true
I20260926 13:50:33.919993 18882 raft_consensus.cc:2454] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 158a311c4c6d4fd0a2b01920fc17df58 in term 0.
I20260926 13:50:33.919993 19162 raft_consensus.cc:2454] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 158a311c4c6d4fd0a2b01920fc17df58 in term 0.
I20260926 13:50:33.920524 18953 leader_election.cc:302] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158a311c4c6d4fd0a2b01920fc17df58, 2f934ba15ebb4ae2a85c023dbff446b5; no voters:
I20260926 13:50:33.920843 19244 raft_consensus.cc:2781] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:33.920943 19244 raft_consensus.cc:482] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:33.921038 19244 raft_consensus.cc:3037] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:33.922108 19244 raft_consensus.cc:504] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.922442 19244 leader_election.cc:288] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [CANDIDATE]: Term 1 election: Requested vote from peers ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943), 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347)
I20260926 13:50:33.922835 19162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6224402742f471286f73bc080ce9780" candidate_uuid: "158a311c4c6d4fd0a2b01920fc17df58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef4305690b9240c784aa1b654e376ddf"
I20260926 13:50:33.922971 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6224402742f471286f73bc080ce9780" candidate_uuid: "158a311c4c6d4fd0a2b01920fc17df58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
I20260926 13:50:33.922961 19162 raft_consensus.cc:3037] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:33.923128 18882 raft_consensus.cc:3037] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:33.924316 18882 raft_consensus.cc:2454] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 158a311c4c6d4fd0a2b01920fc17df58 in term 1.
I20260926 13:50:33.924347 19162 raft_consensus.cc:2454] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 158a311c4c6d4fd0a2b01920fc17df58 in term 1.
I20260926 13:50:33.924729 18953 leader_election.cc:302] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158a311c4c6d4fd0a2b01920fc17df58, 2f934ba15ebb4ae2a85c023dbff446b5; no voters:
I20260926 13:50:33.924991 19244 raft_consensus.cc:2781] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:33.925282 19244 raft_consensus.cc:686] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 LEADER]: Becoming Leader. State: Replica: 158a311c4c6d4fd0a2b01920fc17df58, State: Running, Role: LEADER
I20260926 13:50:33.925686 19244 consensus_queue.cc:237] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } }
I20260926 13:50:33.929303 18724 catalog_manager.cc:5495] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "158a311c4c6d4fd0a2b01920fc17df58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:33.945170 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:33.941566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.10.236.194:33189)
0926 13:50:33.941696 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:33.941701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:33.941716 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:33.941809 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:33.941811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:33.941864 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:33.941979 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:33.942225 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.942735 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.943701 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:33.943926 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:33.944053 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:33.944341 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:33.944781 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:33.944909 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:33.944966 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 13:50:33.945022 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:50:33.951818 18882 raft_consensus.cc:1261] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Refusing update from remote peer 158a311c4c6d4fd0a2b01920fc17df58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:33.951815 19162 raft_consensus.cc:1261] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Refusing update from remote peer 158a311c4c6d4fd0a2b01920fc17df58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:33.952596 19244 consensus_queue.cc:1035] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:33.952828 19246 consensus_queue.cc:1035] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:33.962149 19251 mvcc.cc:205] Tried to move back new op lower bound from 7333603876658421760 to 7333603876562657280. Current Snapshot: MvccSnapshot[applied={T|T < 7333603876658421760}]
I20260926 13:50:33.962510 19250 mvcc.cc:205] Tried to move back new op lower bound from 7333603876658421760 to 7333603876562657280. Current Snapshot: MvccSnapshot[applied={T|T < 7333603876658421760}]
I20260926 13:50:33.968313 18723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:33.968510 18723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:33.970942 18723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Sending DeleteTablet for 3 replicas of tablet c6224402742f471286f73bc080ce9780
I20260926 13:50:33.971854 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet c6224402742f471286f73bc080ce9780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:33 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51196
I20260926 13:50:33.971884 19142 tablet_service.cc:1514] Processing DeleteTablet for tablet c6224402742f471286f73bc080ce9780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:33 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:35976
I20260926 13:50:33.972020 18862 tablet_service.cc:1514] Processing DeleteTablet for tablet c6224402742f471286f73bc080ce9780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:33 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39218
I20260926 13:50:33.972390 19258 tablet_replica.cc:330] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: stopping tablet replica
I20260926 13:50:33.972409 19260 tablet_replica.cc:330] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: stopping tablet replica
I20260926 13:50:33.972704 19258 raft_consensus.cc:2229] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:33.972705 19260 raft_consensus.cc:2229] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:33.973035 19258 raft_consensus.cc:2258] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:33.973744 18723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 13:50:33.974030 19260 raft_consensus.cc:2258] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 13:50:33.974251 18723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:33.974759 19258 ts_tablet_manager.cc:1905] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:33.975530 19259 tablet_replica.cc:330] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: stopping tablet replica
I20260926 13:50:33.976774 19260 ts_tablet_manager.cc:1905] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:33.978305 19258 ts_tablet_manager.cc:1918] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:33.978436 19258 log.cc:1196] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/c6224402742f471286f73bc080ce9780
I20260926 13:50:33.978758 19258 ts_tablet_manager.cc:1939] T c6224402742f471286f73bc080ce9780 P ef4305690b9240c784aa1b654e376ddf: Deleting consensus metadata
I20260926 13:50:33.979511 19260 ts_tablet_manager.cc:1918] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:33.979609 19260 log.cc:1196] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/c6224402742f471286f73bc080ce9780
I20260926 13:50:33.979924 19260 ts_tablet_manager.cc:1939] T c6224402742f471286f73bc080ce9780 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting consensus metadata
I20260926 13:50:33.980840 19259 raft_consensus.cc:2229] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:33.981012 18862 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 (DEFAULT_TABLE table=test-table [id=e290cfa36f174f08a115220a94483fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:33.981276 18862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.981304 19259 raft_consensus.cc:2258] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:33.983037 19259 ts_tablet_manager.cc:1905] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:33.983418 19236 tablet_bootstrap.cc:492] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Bootstrap starting.
I20260926 13:50:33.983623 18709 catalog_manager.cc:4841] TS 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347): tablet c6224402742f471286f73bc080ce9780 (table test-table [id=1a863e95f58342b08b006097042426c1]) successfully deleted
I20260926 13:50:33.984445 19236 tablet_bootstrap.cc:654] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.984748 19002 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 (DEFAULT_TABLE table=test-table [id=e290cfa36f174f08a115220a94483fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:33.985143 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.986171 19259 ts_tablet_manager.cc:1918] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:33.986297 19259 log.cc:1196] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/c6224402742f471286f73bc080ce9780
I20260926 13:50:33.986603 19259 ts_tablet_manager.cc:1939] T c6224402742f471286f73bc080ce9780 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting consensus metadata
I20260926 13:50:33.987454 19237 tablet_bootstrap.cc:492] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Bootstrap starting.
I20260926 13:50:33.987845 18712 catalog_manager.cc:4841] TS 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189): tablet c6224402742f471286f73bc080ce9780 (table test-table [id=1a863e95f58342b08b006097042426c1]) successfully deleted
I20260926 13:50:33.988363 19236 tablet_bootstrap.cc:492] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: No bootstrap required, opened a new log
I20260926 13:50:33.988444 19236 ts_tablet_manager.cc:1397] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 13:50:33.988559 19237 tablet_bootstrap.cc:654] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.988611 19142 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 (DEFAULT_TABLE table=test-table [id=e290cfa36f174f08a115220a94483fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:33.988904 19142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:33.989302 19236 raft_consensus.cc:348] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:33.989427 19236 raft_consensus.cc:374] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.989463 19236 raft_consensus.cc:729] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f934ba15ebb4ae2a85c023dbff446b5, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.989601 19236 consensus_queue.cc:260] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:33.990585 19236 ts_tablet_manager.cc:1428] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:33.991353 19238 tablet_bootstrap.cc:492] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Bootstrap starting.
I20260926 13:50:33.992029 18709 catalog_manager.cc:4841] TS ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943): tablet c6224402742f471286f73bc080ce9780 (table test-table [id=1a863e95f58342b08b006097042426c1]) successfully deleted
I20260926 13:50:33.992427 19238 tablet_bootstrap.cc:654] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:33.993016 19237 tablet_bootstrap.cc:492] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: No bootstrap required, opened a new log
I20260926 13:50:33.993100 19237 ts_tablet_manager.cc:1397] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260926 13:50:33.993670 19238 tablet_bootstrap.cc:492] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: No bootstrap required, opened a new log
I20260926 13:50:33.993587 19237 raft_consensus.cc:348] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:33.993739 19237 raft_consensus.cc:374] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.993777 19238 ts_tablet_manager.cc:1397] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:33.993796 19237 raft_consensus.cc:729] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158a311c4c6d4fd0a2b01920fc17df58, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.993937 19237 consensus_queue.cc:260] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:33.994308 19237 ts_tablet_manager.cc:1428] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:33.994251 19238 raft_consensus.cc:348] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:33.994359 19238 raft_consensus.cc:374] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:33.994390 19238 raft_consensus.cc:729] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Initialized, Role: FOLLOWER
I20260926 13:50:33.994521 19238 consensus_queue.cc:260] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:33.995620 19238 ts_tablet_manager.cc:1428] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260926 13:50:34.019265 19209 tablet.cc:2369] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:34.042985 19243 raft_consensus.cc:482] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:34.043135 19243 raft_consensus.cc:504] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.044373 19243 leader_election.cc:288] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347), 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
W20260926 13:50:34.046064 19069 tablet.cc:2369] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:34.049680 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.044612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:42347 (local address 127.10.236.195:56821)
0926 13:50:34.045035 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.045050 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.045136 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.045689 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.045692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.045708 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:34.045814 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.045819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.046798 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.046802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.048078 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.048088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.049064 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.049068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.049374 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.049400 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.049470 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.049541 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":185}
I20260926 13:50:34.050163 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.044940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:56821 (local address 127.10.236.193:42347)
0926 13:50:34.045071 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.045076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.045112 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:50:34.045466 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.045469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.045516 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.045581 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:34.045933 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.046675 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.048212 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.048813 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.048953 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.049008 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.049973 (+ 965us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.050049 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:50:34.050572 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7b8ed4ee334aa19dcd10fa294e7f11" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" is_pre_election: true
I20260926 13:50:34.050732 18882 raft_consensus.cc:2454] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 0.
I20260926 13:50:34.051043 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.045283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:40845 (local address 127.10.236.194:33189)
0926 13:50:34.045395 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.045399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.045507 (+ 108us) server_negotiation.cc:407] Connection header received
0926 13:50:34.045732 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.045736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.045799 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.045852 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:34.046999 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.047769 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.049324 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.049924 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.050022 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.050082 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.050831 (+ 749us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.050915 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:50:34.051178 19093 leader_election.cc:302] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f934ba15ebb4ae2a85c023dbff446b5, ef4305690b9240c784aa1b654e376ddf; no voters:
I20260926 13:50:34.051589 19243 raft_consensus.cc:2781] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:34.051636 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.044686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33189 (local address 127.10.236.195:40845)
0926 13:50:34.045410 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.045425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.045583 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.046681 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.046684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.046707 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:34.046861 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.046868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.047921 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.047925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.049175 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.049186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.050507 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.050511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.050606 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.050629 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.051358 (+ 729us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.051441 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260926 13:50:34.051651 19243 raft_consensus.cc:482] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:34.052286 19243 raft_consensus.cc:3037] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.052341 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7b8ed4ee334aa19dcd10fa294e7f11" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58" is_pre_election: true
I20260926 13:50:34.052563 19022 raft_consensus.cc:2454] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 0.
I20260926 13:50:34.053722 19243 raft_consensus.cc:504] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.054136 19243 leader_election.cc:288] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 election: Requested vote from peers 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347), 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
I20260926 13:50:34.054502 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7b8ed4ee334aa19dcd10fa294e7f11" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
I20260926 13:50:34.054637 18882 raft_consensus.cc:3037] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.054641 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7b8ed4ee334aa19dcd10fa294e7f11" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58"
I20260926 13:50:34.054834 19022 raft_consensus.cc:3037] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.055820 18882 raft_consensus.cc:2454] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 1.
I20260926 13:50:34.056142 19022 raft_consensus.cc:2454] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 1.
I20260926 13:50:34.056160 19093 leader_election.cc:302] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f934ba15ebb4ae2a85c023dbff446b5, ef4305690b9240c784aa1b654e376ddf; no voters:
I20260926 13:50:34.056476 19243 raft_consensus.cc:2781] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:34.056720 19243 raft_consensus.cc:686] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 1 LEADER]: Becoming Leader. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Running, Role: LEADER
I20260926 13:50:34.057001 19243 consensus_queue.cc:237] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.059620 18723 catalog_manager.cc:5495] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf reported cstate change: term changed from 0 to 1, leader changed from <none> to ef4305690b9240c784aa1b654e376ddf (127.10.236.195). New cstate: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } health_report { overall_health: HEALTHY } } }
W20260926 13:50:34.074661 18929 tablet.cc:2369] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 13:50:34 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430632, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.254@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:34.094230 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.085293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33855 (local address 127.0.0.1:57336)
0926 13:50:34.085601 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.085618 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.085771 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.086043 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.086045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.086453 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:34.086559 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.086566 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.087308 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.087312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.087903 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.087911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.088394 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.088397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.088470 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.089083 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:34.089095 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:34.090680 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:34.092607 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:34.092617 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:34.092624 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:34.092870 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:34.093151 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:34.093157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:34.093159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:34.093240 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:34.093625 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:34.093634 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:34.093740 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.093867 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.094029 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":115,"threads_started":1}
I20260926 13:50:34.094229 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.085428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.10.236.254:33855)
0926 13:50:34.085581 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.085586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.085709 (+ 123us) server_negotiation.cc:407] Connection header received
0926 13:50:34.085893 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.085895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.085941 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.086027 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:34.086679 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.087190 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.088021 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.088233 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.088388 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.090818 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:34.090822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:34.090824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:34.090854 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:34.092486 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:34.092971 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:34.092977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:34.092979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:34.093063 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:34.093330 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:34.093335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:34.093336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:34.093517 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:34.093617 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.093909 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.094103 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":57}
I20260926 13:50:34.099274 18723 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:57336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 13:50:34.100142 18723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:34.105549 18862 tablet_service.cc:1467] Processing CreateTablet for tablet cf36327759284ff09cee91efa489b6c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=37ca85ad2deb4c62bccf8aa84b2f4346]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:34.106004 18862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf36327759284ff09cee91efa489b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.107854 19142 tablet_service.cc:1467] Processing CreateTablet for tablet cf36327759284ff09cee91efa489b6c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=37ca85ad2deb4c62bccf8aa84b2f4346]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:34.108294 19142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf36327759284ff09cee91efa489b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.109272 19236 tablet_bootstrap.cc:492] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: Bootstrap starting.
I20260926 13:50:34.110304 19236 tablet_bootstrap.cc:654] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.111119 19238 tablet_bootstrap.cc:492] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: Bootstrap starting.
I20260926 13:50:34.111543 19002 tablet_service.cc:1467] Processing CreateTablet for tablet cf36327759284ff09cee91efa489b6c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=37ca85ad2deb4c62bccf8aa84b2f4346]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:50:34.111898 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf36327759284ff09cee91efa489b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.112083 19238 tablet_bootstrap.cc:654] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.113792 19238 tablet_bootstrap.cc:492] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: No bootstrap required, opened a new log
I20260926 13:50:34.113867 19238 ts_tablet_manager.cc:1397] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:50:34.114342 19238 raft_consensus.cc:348] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.114463 19238 raft_consensus.cc:374] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.114526 19238 raft_consensus.cc:729] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.114655 19238 consensus_queue.cc:260] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.114892 19243 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.115020 19243 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.115268 19238 ts_tablet_manager.cc:1428] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260926 13:50:34.116453 19237 tablet_bootstrap.cc:492] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: Bootstrap starting.
I20260926 13:50:34.117487 19237 tablet_bootstrap.cc:654] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.117586 19236 tablet_bootstrap.cc:492] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: No bootstrap required, opened a new log
I20260926 13:50:34.117654 19236 ts_tablet_manager.cc:1397] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260926 13:50:34.118122 19236 raft_consensus.cc:348] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.118247 19236 raft_consensus.cc:374] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.118270 19236 raft_consensus.cc:729] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f934ba15ebb4ae2a85c023dbff446b5, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.118431 19236 consensus_queue.cc:260] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.118724 19236 ts_tablet_manager.cc:1428] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:34.118673 19242 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.118821 19242 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.124476 19237 tablet_bootstrap.cc:492] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: No bootstrap required, opened a new log
I20260926 13:50:34.124594 19237 ts_tablet_manager.cc:1397] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260926 13:50:34.125142 19237 raft_consensus.cc:348] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.125283 19237 raft_consensus.cc:374] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.125322 19237 raft_consensus.cc:729] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158a311c4c6d4fd0a2b01920fc17df58, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.125463 19237 consensus_queue.cc:260] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.125716 19244 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.125850 19244 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.126113 19237 ts_tablet_manager.cc:1428] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 13:50:34.280544 19243 raft_consensus.cc:482] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:34.280709 19243 raft_consensus.cc:504] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.281198 19243 leader_election.cc:288] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347), 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
I20260926 13:50:34.281760 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36327759284ff09cee91efa489b6c1" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" is_pre_election: true
I20260926 13:50:34.281760 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36327759284ff09cee91efa489b6c1" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58" is_pre_election: true
I20260926 13:50:34.281926 18882 raft_consensus.cc:2454] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 0.
I20260926 13:50:34.281939 19022 raft_consensus.cc:2454] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 0.
I20260926 13:50:34.282287 19096 leader_election.cc:302] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158a311c4c6d4fd0a2b01920fc17df58, ef4305690b9240c784aa1b654e376ddf; no voters:
I20260926 13:50:34.282552 19243 raft_consensus.cc:2781] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:34.282611 19243 raft_consensus.cc:482] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:34.282639 19243 raft_consensus.cc:3037] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.283802 19243 raft_consensus.cc:504] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.284148 19243 leader_election.cc:288] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 election: Requested vote from peers 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347), 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
I20260926 13:50:34.284569 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36327759284ff09cee91efa489b6c1" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58"
I20260926 13:50:34.284605 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf36327759284ff09cee91efa489b6c1" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
I20260926 13:50:34.284710 19022 raft_consensus.cc:3037] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.284727 18882 raft_consensus.cc:3037] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.285611 18882 raft_consensus.cc:2454] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 1.
I20260926 13:50:34.285867 19022 raft_consensus.cc:2454] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 1.
I20260926 13:50:34.285919 19093 leader_election.cc:302] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f934ba15ebb4ae2a85c023dbff446b5, ef4305690b9240c784aa1b654e376ddf; no voters:
I20260926 13:50:34.286085 19243 raft_consensus.cc:2781] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:34.286211 19243 raft_consensus.cc:686] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [term 1 LEADER]: Becoming Leader. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Running, Role: LEADER
I20260926 13:50:34.286355 19243 consensus_queue.cc:237] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.287006 19265 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: TxnStatusTablet state changed. Reason: New leader ef4305690b9240c784aa1b654e376ddf. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.287127 19265 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:34.287355 19277 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 13:50:34.287547 19277 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 13:50:34.288173 18724 catalog_manager.cc:5495] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf reported cstate change: term changed from 0 to 1, leader changed from <none> to ef4305690b9240c784aa1b654e376ddf (127.10.236.195). New cstate: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } health_report { overall_health: HEALTHY } } }
I20260926 13:50:34.313400 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.308995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.10.236.195:35943)
0926 13:50:34.309124 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.309128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.309249 (+ 121us) server_negotiation.cc:407] Connection header received
0926 13:50:34.309449 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.309452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.309505 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.309586 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:34.309895 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.310426 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.311718 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.311945 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.312032 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.312430 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:34.312957 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:34.313050 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.313197 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.313247 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:50:34.313401 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.308897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35943 (local address 127.0.0.1:33032)
0926 13:50:34.309157 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.309174 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.309298 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.309628 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.309631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.309640 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:34.309785 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.309792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.310532 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.310536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.311602 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.311614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.312061 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.312065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.312162 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.312219 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:34.313056 (+ 837us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:34.313074 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.313158 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.313248 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260926 13:50:34.316659 18882 raft_consensus.cc:1261] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Refusing update from remote peer ef4305690b9240c784aa1b654e376ddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:34.317188 19243 consensus_queue.cc:1035] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:34.317312 19022 raft_consensus.cc:1261] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Refusing update from remote peer ef4305690b9240c784aa1b654e376ddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:34.317849 19243 consensus_queue.cc:1035] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:34.318751 19242 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: TxnStatusTablet state changed. Reason: New leader ef4305690b9240c784aa1b654e376ddf. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.318882 19242 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.320124 19244 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: TxnStatusTablet state changed. Reason: New leader ef4305690b9240c784aa1b654e376ddf. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.320254 19244 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.321313 19265 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.321424 19265 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:34.322216 19242 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.322336 19242 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P 2f934ba15ebb4ae2a85c023dbff446b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.328178 19243 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.328325 19243 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P ef4305690b9240c784aa1b654e376ddf: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:50:34.329188 19249 mvcc.cc:205] Tried to move back new op lower bound from 7333603878155751424 to 7333603878038286336. Current Snapshot: MvccSnapshot[applied={T|T < 7333603878155751424}]
I20260926 13:50:34.329523 19244 tablet_replica.cc:439] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } } }
I20260926 13:50:34.329643 19244 tablet_replica.cc:442] T cf36327759284ff09cee91efa489b6c1 P 158a311c4c6d4fd0a2b01920fc17df58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:50:34.334448 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.331255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.10.236.195:35943)
0926 13:50:34.331375 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.331380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.331592 (+ 212us) server_negotiation.cc:407] Connection header received
0926 13:50:34.331728 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.331731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.331767 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.331861 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:34.332105 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.332612 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.333433 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.333660 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.333763 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.334019 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:34.334093 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:34.334172 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.334265 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.334318 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:50:34.347101 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.341566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:49915 (local address 127.10.236.195:35943)
0926 13:50:34.341691 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.341695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.343823 (+ 2128us) server_negotiation.cc:407] Connection header received
0926 13:50:34.343982 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.343984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.344068 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.344131 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:34.344470 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.345101 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.346072 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.346296 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.346398 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.346655 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:34.346729 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:34.346803 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.346890 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.346925 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:50:34.347126 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.341517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35943 (local address 127.10.236.195:49915)
0926 13:50:34.343739 (+ 2222us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.343753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.343872 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.344252 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.344255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.344260 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:34.344346 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.344351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.345206 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.345209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.345943 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.345952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.346397 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.346400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.346497 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.346533 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:34.346806 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:34.346814 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.346873 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.346938 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2100,"thread_start_us":114,"threads_started":1}
I20260926 13:50:34.353732 18882 raft_consensus.cc:1261] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Refusing update from remote peer ef4305690b9240c784aa1b654e376ddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:34.354105 19022 raft_consensus.cc:1261] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Refusing update from remote peer ef4305690b9240c784aa1b654e376ddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:34.354162 19243 consensus_queue.cc:1035] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:34.354645 19243 consensus_queue.cc:1035] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:34.392769 18724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:34.392949 18724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:34.395192 18724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Sending DeleteTablet for 3 replicas of tablet 2f7b8ed4ee334aa19dcd10fa294e7f11
I20260926 13:50:34.395891 18862 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:34 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39218
I20260926 13:50:34.395852 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:34 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51196
I20260926 13:50:34.395900 19142 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:34 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:35976
I20260926 13:50:34.396073 19260 tablet_replica.cc:330] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: stopping tablet replica
I20260926 13:50:34.396082 19258 tablet_replica.cc:330] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: stopping tablet replica
I20260926 13:50:34.396095 19259 tablet_replica.cc:330] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: stopping tablet replica
I20260926 13:50:34.396183 19260 raft_consensus.cc:2229] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:34.396183 19258 raft_consensus.cc:2229] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:34.396202 19259 raft_consensus.cc:2229] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:34.396293 19260 raft_consensus.cc:2258] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:34.396364 19259 raft_consensus.cc:2258] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:34.396478 19258 raft_consensus.cc:2258] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:34.397185 19260 ts_tablet_manager.cc:1905] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:34.397262 19259 ts_tablet_manager.cc:1905] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:34.400039 19258 ts_tablet_manager.cc:1905] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:34.400112 19260 ts_tablet_manager.cc:1918] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:34.400156 19259 ts_tablet_manager.cc:1918] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:34.400185 19260 log.cc:1196] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/2f7b8ed4ee334aa19dcd10fa294e7f11
I20260926 13:50:34.400221 19259 log.cc:1196] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/2f7b8ed4ee334aa19dcd10fa294e7f11
I20260926 13:50:34.400466 19259 ts_tablet_manager.cc:1939] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting consensus metadata
I20260926 13:50:34.400466 19260 ts_tablet_manager.cc:1939] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting consensus metadata
I20260926 13:50:34.401479 18712 catalog_manager.cc:4841] TS 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189): tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 (table test-table [id=e290cfa36f174f08a115220a94483fe8]) successfully deleted
I20260926 13:50:34.402745 19258 ts_tablet_manager.cc:1918] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:34.402843 19258 log.cc:1196] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/2f7b8ed4ee334aa19dcd10fa294e7f11
I20260926 13:50:34.403110 19258 ts_tablet_manager.cc:1939] T 2f7b8ed4ee334aa19dcd10fa294e7f11 P ef4305690b9240c784aa1b654e376ddf: Deleting consensus metadata
I20260926 13:50:34.403867 18709 catalog_manager.cc:4841] TS ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943): tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 (table test-table [id=e290cfa36f174f08a115220a94483fe8]) successfully deleted
I20260926 13:50:34.405138 18709 catalog_manager.cc:4841] TS 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347): tablet 2f7b8ed4ee334aa19dcd10fa294e7f11 (table test-table [id=e290cfa36f174f08a115220a94483fe8]) successfully deleted
Sep 26 13:50:34 dist-test-slave-r34j krb5kdc[18680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:50:34 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:34 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:50:34.429550 18774 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:34.422538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.10.236.254:33855)
0926 13:50:34.422678 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.422683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.422858 (+ 175us) server_negotiation.cc:407] Connection header received
0926 13:50:34.423043 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.423047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.423103 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.423195 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:34.424053 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.424588 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.425461 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.425691 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.425805 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.429414 (+ 3609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45216: BlockingRecv error: recv got EOF from 127.0.0.1:45216 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 13:50:34 dist-test-slave-r34j krb5kdc[18680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM
I20260926 13:50:34.441478 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.431274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.10.236.254:33855)
0926 13:50:34.431390 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.431394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.431630 (+ 236us) server_negotiation.cc:407] Connection header received
0926 13:50:34.431761 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.431763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.431808 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.431892 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:34.432680 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.433364 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.434521 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.434745 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.434840 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.437583 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:34.437587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:34.437589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:34.437615 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:34.439382 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:34.439954 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:34.439958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:34.439960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:34.440053 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:34.440407 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:34.440411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:34.440413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:34.440624 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:34.440780 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.441073 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.441203 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:50:34.444468 18724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:34.445027 18724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:34.450080 18862 tablet_service.cc:1467] Processing CreateTablet for tablet 267de853a87c4fccb4fc1abba34a567a (DEFAULT_TABLE table=test-table [id=a596e0dc28994ed798e2e5aa64dd76d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:34.450198 19002 tablet_service.cc:1467] Processing CreateTablet for tablet 267de853a87c4fccb4fc1abba34a567a (DEFAULT_TABLE table=test-table [id=a596e0dc28994ed798e2e5aa64dd76d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:34.450392 18862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267de853a87c4fccb4fc1abba34a567a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.450489 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267de853a87c4fccb4fc1abba34a567a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.452517 19237 tablet_bootstrap.cc:492] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Bootstrap starting.
I20260926 13:50:34.452895 19142 tablet_service.cc:1467] Processing CreateTablet for tablet 267de853a87c4fccb4fc1abba34a567a (DEFAULT_TABLE table=test-table [id=a596e0dc28994ed798e2e5aa64dd76d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:34.453156 19142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267de853a87c4fccb4fc1abba34a567a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.453364 19237 tablet_bootstrap.cc:654] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.454522 19236 tablet_bootstrap.cc:492] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Bootstrap starting.
I20260926 13:50:34.454672 19237 tablet_bootstrap.cc:492] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: No bootstrap required, opened a new log
I20260926 13:50:34.454782 19237 ts_tablet_manager.cc:1397] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:34.455160 19237 raft_consensus.cc:348] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.455277 19237 raft_consensus.cc:374] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.455317 19237 raft_consensus.cc:729] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158a311c4c6d4fd0a2b01920fc17df58, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.455351 19236 tablet_bootstrap.cc:654] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.455443 19237 consensus_queue.cc:260] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.455745 19237 ts_tablet_manager.cc:1428] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:34.456908 19236 tablet_bootstrap.cc:492] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: No bootstrap required, opened a new log
I20260926 13:50:34.456948 19238 tablet_bootstrap.cc:492] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Bootstrap starting.
I20260926 13:50:34.456996 19236 ts_tablet_manager.cc:1397] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 13:50:34.457429 19236 raft_consensus.cc:348] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.457553 19236 raft_consensus.cc:374] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.457587 19236 raft_consensus.cc:729] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f934ba15ebb4ae2a85c023dbff446b5, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.457710 19236 consensus_queue.cc:260] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.457894 19238 tablet_bootstrap.cc:654] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.458004 19236 ts_tablet_manager.cc:1428] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 13:50:34.459193 19238 tablet_bootstrap.cc:492] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: No bootstrap required, opened a new log
I20260926 13:50:34.459275 19238 ts_tablet_manager.cc:1397] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:34.459699 19238 raft_consensus.cc:348] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.459810 19238 raft_consensus.cc:374] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.459852 19238 raft_consensus.cc:729] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.460007 19238 consensus_queue.cc:260] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.460314 19238 ts_tablet_manager.cc:1428] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:34.690837 19287 raft_consensus.cc:482] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:34.690982 19287 raft_consensus.cc:504] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.691447 19287 leader_election.cc:288] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347), 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
I20260926 13:50:34.691953 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267de853a87c4fccb4fc1abba34a567a" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" is_pre_election: true
I20260926 13:50:34.691955 19021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267de853a87c4fccb4fc1abba34a567a" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58" is_pre_election: true
I20260926 13:50:34.692124 18882 raft_consensus.cc:2454] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 0.
I20260926 13:50:34.692139 19021 raft_consensus.cc:2454] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 0.
I20260926 13:50:34.692520 19096 leader_election.cc:302] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 158a311c4c6d4fd0a2b01920fc17df58, 2f934ba15ebb4ae2a85c023dbff446b5, ef4305690b9240c784aa1b654e376ddf; no voters:
I20260926 13:50:34.692802 19287 raft_consensus.cc:2781] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:34.692873 19287 raft_consensus.cc:482] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:34.692929 19287 raft_consensus.cc:3037] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.694291 19287 raft_consensus.cc:504] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.694774 19287 leader_election.cc:288] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 election: Requested vote from peers 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347), 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189)
I20260926 13:50:34.695178 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267de853a87c4fccb4fc1abba34a567a" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f934ba15ebb4ae2a85c023dbff446b5"
I20260926 13:50:34.695250 19021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267de853a87c4fccb4fc1abba34a567a" candidate_uuid: "ef4305690b9240c784aa1b654e376ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58"
I20260926 13:50:34.695304 18882 raft_consensus.cc:3037] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.695410 19021 raft_consensus.cc:3037] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.696264 18882 raft_consensus.cc:2454] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 1.
I20260926 13:50:34.696491 19021 raft_consensus.cc:2454] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4305690b9240c784aa1b654e376ddf in term 1.
I20260926 13:50:34.696569 19093 leader_election.cc:302] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f934ba15ebb4ae2a85c023dbff446b5, ef4305690b9240c784aa1b654e376ddf; no voters:
I20260926 13:50:34.696743 19287 raft_consensus.cc:2781] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:34.696857 19287 raft_consensus.cc:686] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 1 LEADER]: Becoming Leader. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Running, Role: LEADER
I20260926 13:50:34.697019 19287 consensus_queue.cc:237] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.698849 18725 catalog_manager.cc:5495] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf reported cstate change: term changed from 0 to 1, leader changed from <none> to ef4305690b9240c784aa1b654e376ddf (127.10.236.195). New cstate: current_term: 1 leader_uuid: "ef4305690b9240c784aa1b654e376ddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } health_report { overall_health: HEALTHY } } }
I20260926 13:50:34.753988 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.750887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.10.236.195:35943)
0926 13:50:34.751051 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.751057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.751073 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:34.751123 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.751125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.751166 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.751250 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:34.751545 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.752068 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.752927 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.753149 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.753260 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.753530 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:34.753600 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:34.753678 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.753771 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.753826 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:50:34.756237 18882 raft_consensus.cc:1261] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Refusing update from remote peer ef4305690b9240c784aa1b654e376ddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:34.756659 19021 raft_consensus.cc:1261] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Refusing update from remote peer ef4305690b9240c784aa1b654e376ddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:34.756729 19265 consensus_queue.cc:1035] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:34.757117 19287 consensus_queue.cc:1035] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:34.766013 18724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:34.766165 18724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:34.768376 18724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Sending DeleteTablet for 3 replicas of tablet 267de853a87c4fccb4fc1abba34a567a
I20260926 13:50:34.769053 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet 267de853a87c4fccb4fc1abba34a567a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:34 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51196
I20260926 13:50:34.769105 18862 tablet_service.cc:1514] Processing DeleteTablet for tablet 267de853a87c4fccb4fc1abba34a567a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:34 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39218
I20260926 13:50:34.769286 19260 tablet_replica.cc:330] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: stopping tablet replica
I20260926 13:50:34.769255 19142 tablet_service.cc:1514] Processing DeleteTablet for tablet 267de853a87c4fccb4fc1abba34a567a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:34 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:35976
I20260926 13:50:34.769444 19258 tablet_replica.cc:330] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: stopping tablet replica
I20260926 13:50:34.769443 19260 raft_consensus.cc:2229] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:34.769550 19258 raft_consensus.cc:2229] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:34.769616 19260 raft_consensus.cc:2258] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:34.769737 19258 raft_consensus.cc:2258] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:34.770488 19260 ts_tablet_manager.cc:1905] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:34.771302 19258 ts_tablet_manager.cc:1905] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:34.772670 19260 ts_tablet_manager.cc:1918] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:34.772773 19260 log.cc:1196] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/267de853a87c4fccb4fc1abba34a567a
I20260926 13:50:34.773037 19260 ts_tablet_manager.cc:1939] T 267de853a87c4fccb4fc1abba34a567a P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting consensus metadata
I20260926 13:50:34.773594 19258 ts_tablet_manager.cc:1918] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:34.773666 19258 log.cc:1196] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/267de853a87c4fccb4fc1abba34a567a
I20260926 13:50:34.773715 18709 catalog_manager.cc:4841] TS 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347): tablet 267de853a87c4fccb4fc1abba34a567a (table test-table [id=a596e0dc28994ed798e2e5aa64dd76d4]) successfully deleted
I20260926 13:50:34.773895 19258 ts_tablet_manager.cc:1939] T 267de853a87c4fccb4fc1abba34a567a P ef4305690b9240c784aa1b654e376ddf: Deleting consensus metadata
I20260926 13:50:34.774411 18724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 13:50:34.774678 18709 catalog_manager.cc:4841] TS ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943): tablet 267de853a87c4fccb4fc1abba34a567a (table test-table [id=a596e0dc28994ed798e2e5aa64dd76d4]) successfully deleted
W20260926 13:50:34.774885 18724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:34.779613 19259 tablet_replica.cc:330] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: stopping tablet replica
I20260926 13:50:34.779778 19259 raft_consensus.cc:2229] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:34.779930 19259 raft_consensus.cc:2258] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:34.779973 19002 tablet_service.cc:1467] Processing CreateTablet for tablet e93ad1ef0b034b11b2674ea746bc8061 (DEFAULT_TABLE table=test-table [id=995cbdcdcf214659bc7e848cd284879f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:34.779989 18862 tablet_service.cc:1467] Processing CreateTablet for tablet e93ad1ef0b034b11b2674ea746bc8061 (DEFAULT_TABLE table=test-table [id=995cbdcdcf214659bc7e848cd284879f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:34.780232 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93ad1ef0b034b11b2674ea746bc8061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.780333 18862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93ad1ef0b034b11b2674ea746bc8061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.782299 19236 tablet_bootstrap.cc:492] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Bootstrap starting.
I20260926 13:50:34.782615 19237 tablet_bootstrap.cc:492] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Bootstrap starting.
I20260926 13:50:34.783363 19236 tablet_bootstrap.cc:654] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.783414 19237 tablet_bootstrap.cc:654] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.784601 19237 tablet_bootstrap.cc:492] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: No bootstrap required, opened a new log
I20260926 13:50:34.784682 19237 ts_tablet_manager.cc:1397] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 13:50:34.785029 19237 raft_consensus.cc:348] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.785143 19237 raft_consensus.cc:374] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.785171 19237 raft_consensus.cc:729] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158a311c4c6d4fd0a2b01920fc17df58, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.785279 19237 consensus_queue.cc:260] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.785527 19237 ts_tablet_manager.cc:1428] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:50:34.785880 19142 tablet_service.cc:1467] Processing CreateTablet for tablet e93ad1ef0b034b11b2674ea746bc8061 (DEFAULT_TABLE table=test-table [id=995cbdcdcf214659bc7e848cd284879f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:34.786155 19142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93ad1ef0b034b11b2674ea746bc8061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:34.788460 19238 tablet_bootstrap.cc:492] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Bootstrap starting.
I20260926 13:50:34.788717 19236 tablet_bootstrap.cc:492] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: No bootstrap required, opened a new log
I20260926 13:50:34.788795 19236 ts_tablet_manager.cc:1397] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260926 13:50:34.789319 19238 tablet_bootstrap.cc:654] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:34.789234 19236 raft_consensus.cc:348] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.789357 19236 raft_consensus.cc:374] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.789387 19236 raft_consensus.cc:729] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f934ba15ebb4ae2a85c023dbff446b5, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.789536 19236 consensus_queue.cc:260] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.789831 19236 ts_tablet_manager.cc:1428] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260926 13:50:34.790407 19238 tablet_bootstrap.cc:492] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: No bootstrap required, opened a new log
I20260926 13:50:34.790488 19238 ts_tablet_manager.cc:1397] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:34.790887 19259 ts_tablet_manager.cc:1905] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:34.790875 19238 raft_consensus.cc:348] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.790990 19238 raft_consensus.cc:374] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:34.791020 19238 raft_consensus.cc:729] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef4305690b9240c784aa1b654e376ddf, State: Initialized, Role: FOLLOWER
I20260926 13:50:34.791141 19238 consensus_queue.cc:260] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.791393 19238 ts_tablet_manager.cc:1428] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:50:34.793506 19259 ts_tablet_manager.cc:1918] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:34.793599 19259 log.cc:1196] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/267de853a87c4fccb4fc1abba34a567a
I20260926 13:50:34.793879 19259 ts_tablet_manager.cc:1939] T 267de853a87c4fccb4fc1abba34a567a P 158a311c4c6d4fd0a2b01920fc17df58: Deleting consensus metadata
I20260926 13:50:34.794523 18712 catalog_manager.cc:4841] TS 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189): tablet 267de853a87c4fccb4fc1abba34a567a (table test-table [id=a596e0dc28994ed798e2e5aa64dd76d4]) successfully deleted
I20260926 13:50:34.973588 19242 raft_consensus.cc:482] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:34.973742 19242 raft_consensus.cc:504] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.975211 19242 leader_election.cc:288] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189), ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943)
I20260926 13:50:34.979528 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.975411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33189 (local address 127.10.236.193:55573)
0926 13:50:34.975857 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.975873 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.975989 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.976276 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.976282 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.976304 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:34.976451 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.976464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.977318 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.977320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.978220 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.978229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.979006 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.979013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.979110 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.979142 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.979221 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.979311 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":2,"thread_start_us":140,"threads_started":1}
I20260926 13:50:34.979538 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.975589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:55573 (local address 127.10.236.194:33189)
0926 13:50:34.976004 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.976012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.976032 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:50:34.976092 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.976095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.976156 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.976246 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:34.976579 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.977162 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.978356 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.978844 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.978950 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.979022 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.979259 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.979351 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":100,"threads_started":1}
I20260926 13:50:34.979605 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.975734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:46329 (local address 127.10.236.195:35943)
0926 13:50:34.975851 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:34.975855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:34.976075 (+ 220us) server_negotiation.cc:407] Connection header received
0926 13:50:34.976177 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:34.976179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:34.976252 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:34.976328 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:34.976643 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.977129 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.978493 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.978926 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.979059 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.979112 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:34.979366 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 13:50:34.979441 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:50:34.979552 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:34.975639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35943 (local address 127.10.236.193:46329)
0926 13:50:34.975953 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:34.975969 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:34.976087 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:34.976373 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:34.976377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:34.976401 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:34.976498 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.976527 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.977275 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.977279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.978264 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:34.978304 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:34.979081 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:34.979084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:34.979162 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:34.979180 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:50:34.979287 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 13:50:34.979354 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":44,"thread_start_us":65,"threads_started":1}
I20260926 13:50:34.980038 19021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93ad1ef0b034b11b2674ea746bc8061" candidate_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58" is_pre_election: true
I20260926 13:50:34.980192 19021 raft_consensus.cc:2454] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f934ba15ebb4ae2a85c023dbff446b5 in term 0.
I20260926 13:50:34.980607 18816 leader_election.cc:302] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158a311c4c6d4fd0a2b01920fc17df58, 2f934ba15ebb4ae2a85c023dbff446b5; no voters:
I20260926 13:50:34.980680 19162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93ad1ef0b034b11b2674ea746bc8061" candidate_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef4305690b9240c784aa1b654e376ddf" is_pre_election: true
I20260926 13:50:34.980863 19162 raft_consensus.cc:2454] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f934ba15ebb4ae2a85c023dbff446b5 in term 0.
I20260926 13:50:34.981014 19242 raft_consensus.cc:2781] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:34.981099 19242 raft_consensus.cc:482] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:34.981163 19242 raft_consensus.cc:3037] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.982244 19242 raft_consensus.cc:504] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.982684 19242 leader_election.cc:288] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [CANDIDATE]: Term 1 election: Requested vote from peers 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189), ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943)
I20260926 13:50:34.983021 19162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93ad1ef0b034b11b2674ea746bc8061" candidate_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef4305690b9240c784aa1b654e376ddf"
I20260926 13:50:34.983044 19021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93ad1ef0b034b11b2674ea746bc8061" candidate_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158a311c4c6d4fd0a2b01920fc17df58"
I20260926 13:50:34.983148 19162 raft_consensus.cc:3037] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.983198 19021 raft_consensus.cc:3037] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:34.984266 19162 raft_consensus.cc:2454] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f934ba15ebb4ae2a85c023dbff446b5 in term 1.
I20260926 13:50:34.984293 19021 raft_consensus.cc:2454] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f934ba15ebb4ae2a85c023dbff446b5 in term 1.
I20260926 13:50:34.984611 18816 leader_election.cc:302] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158a311c4c6d4fd0a2b01920fc17df58, 2f934ba15ebb4ae2a85c023dbff446b5; no voters:
I20260926 13:50:34.984798 19242 raft_consensus.cc:2781] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:34.984997 19242 raft_consensus.cc:686] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f934ba15ebb4ae2a85c023dbff446b5, State: Running, Role: LEADER
I20260926 13:50:34.985467 19242 consensus_queue.cc:237] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } }
I20260926 13:50:34.988409 18724 catalog_manager.cc:5495] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193). New cstate: current_term: 1 leader_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f934ba15ebb4ae2a85c023dbff446b5" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 42347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 } health_report { overall_health: UNKNOWN } } }
I20260926 13:50:35.046644 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:35.041676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.10.236.193:42347)
0926 13:50:35.042028 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:35.042039 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:35.042052 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:50:35.042127 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:35.042129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:35.042181 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:35.042501 (+ 320us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:35.042804 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.043543 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.044874 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.045154 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.045250 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:35.045563 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:35.046084 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:35.046210 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:35.046295 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 13:50:35.046359 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":180,"threads_started":1}
I20260926 13:50:35.058012 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:35.054632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44925 (local address 127.10.236.195:35943)
0926 13:50:35.054781 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:35.054786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:35.054800 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:35.054948 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:35.054952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:35.055011 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:35.055120 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:35.055388 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.056009 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.056899 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.057134 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.057244 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:35.057554 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:35.057633 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:35.057715 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:35.057809 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:50:35.057861 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:50:35.058029 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:35.054402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35943 (local address 127.10.236.193:44925)
0926 13:50:35.054708 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:35.054724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:35.054828 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:35.055131 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:35.055134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:35.055142 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:35.055279 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:35.055284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.056117 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.056123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:35.056774 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:35.056782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.057276 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.057282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:35.057369 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:35.057410 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:35.057719 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:35.057727 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:50:35.057789 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 13:50:35.057869 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260926 13:50:35.064220 19021 raft_consensus.cc:1261] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Refusing update from remote peer 2f934ba15ebb4ae2a85c023dbff446b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:35.064471 19162 raft_consensus.cc:1261] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Refusing update from remote peer 2f934ba15ebb4ae2a85c023dbff446b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:35.064738 19314 consensus_queue.cc:1035] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "158a311c4c6d4fd0a2b01920fc17df58" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:35.065347 19314 consensus_queue.cc:1035] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef4305690b9240c784aa1b654e376ddf" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:35.082057 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:35.078616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:60177 (local address 127.10.236.193:42347)
0926 13:50:35.078735 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:35.078739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:35.078755 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:35.078821 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:35.078824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:35.078870 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:35.078966 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:35.079252 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.079938 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.080888 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.081146 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.081267 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:35.081577 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:35.081664 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:35.081749 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:35.081850 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 13:50:35.081905 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:50:35.082079 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:35.078369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:42347 (local address 127.10.236.195:60177)
0926 13:50:35.078605 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:35.078622 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:35.078721 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:35.078988 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:35.078991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:35.078999 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:50:35.079138 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:35.079143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.080052 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.080059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:35.080754 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:35.080763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:35.081266 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:35.081269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:35.081399 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:35.081426 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:50:35.081754 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:50:35.081764 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 13:50:35.081838 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 13:50:35.081934 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260926 13:50:35.108816 18724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:35.109006 18724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:35.111635 18724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35cad7b7ffa471e81fbf9f32517e4f4: Sending DeleteTablet for 3 replicas of tablet e93ad1ef0b034b11b2674ea746bc8061
I20260926 13:50:35.112493 19142 tablet_service.cc:1514] Processing DeleteTablet for tablet e93ad1ef0b034b11b2674ea746bc8061 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:35 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:35976
I20260926 13:50:35.112468 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet e93ad1ef0b034b11b2674ea746bc8061 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:35 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51196
I20260926 13:50:35.112494 18862 tablet_service.cc:1514] Processing DeleteTablet for tablet e93ad1ef0b034b11b2674ea746bc8061 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:35 UTC) from {username='oryx', principal='oryx/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39218
I20260926 13:50:35.112696 19258 tablet_replica.cc:330] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: stopping tablet replica
I20260926 13:50:35.112697 19259 tablet_replica.cc:330] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: stopping tablet replica
I20260926 13:50:35.112709 19260 tablet_replica.cc:330] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: stopping tablet replica
I20260926 13:50:35.112814 19258 raft_consensus.cc:2229] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:35.112814 19259 raft_consensus.cc:2229] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:35.112929 19259 raft_consensus.cc:2258] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:35.112929 19258 raft_consensus.cc:2258] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:35.112917 19260 raft_consensus.cc:2229] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:35.113267 19260 raft_consensus.cc:2258] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:35.113801 19259 ts_tablet_manager.cc:1905] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:35.113940 19258 ts_tablet_manager.cc:1905] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:35.114120 19260 ts_tablet_manager.cc:1905] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:35.116587 19258 ts_tablet_manager.cc:1918] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:35.116587 19259 ts_tablet_manager.cc:1918] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:35.116662 19259 log.cc:1196] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/e93ad1ef0b034b11b2674ea746bc8061
I20260926 13:50:35.116662 19258 log.cc:1196] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/e93ad1ef0b034b11b2674ea746bc8061
I20260926 13:50:35.116770 19260 ts_tablet_manager.cc:1918] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:50:35.116840 19260 log.cc:1196] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/e93ad1ef0b034b11b2674ea746bc8061
I20260926 13:50:35.116925 19258 ts_tablet_manager.cc:1939] T e93ad1ef0b034b11b2674ea746bc8061 P ef4305690b9240c784aa1b654e376ddf: Deleting consensus metadata
I20260926 13:50:35.116925 19259 ts_tablet_manager.cc:1939] T e93ad1ef0b034b11b2674ea746bc8061 P 158a311c4c6d4fd0a2b01920fc17df58: Deleting consensus metadata
I20260926 13:50:35.117098 19260 ts_tablet_manager.cc:1939] T e93ad1ef0b034b11b2674ea746bc8061 P 2f934ba15ebb4ae2a85c023dbff446b5: Deleting consensus metadata
I20260926 13:50:35.117914 18709 catalog_manager.cc:4841] TS 2f934ba15ebb4ae2a85c023dbff446b5 (127.10.236.193:42347): tablet e93ad1ef0b034b11b2674ea746bc8061 (table test-table [id=995cbdcdcf214659bc7e848cd284879f]) successfully deleted
I20260926 13:50:35.118017 18712 catalog_manager.cc:4841] TS 158a311c4c6d4fd0a2b01920fc17df58 (127.10.236.194:33189): tablet e93ad1ef0b034b11b2674ea746bc8061 (table test-table [id=995cbdcdcf214659bc7e848cd284879f]) successfully deleted
I20260926 13:50:35.118335 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 18797
I20260926 13:50:35.119637 18709 catalog_manager.cc:4841] TS ef4305690b9240c784aa1b654e376ddf (127.10.236.195:35943): tablet e93ad1ef0b034b11b2674ea746bc8061 (table test-table [id=995cbdcdcf214659bc7e848cd284879f]) successfully deleted
I20260926 13:50:35.125686 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 18937
I20260926 13:50:35.132573 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19077
I20260926 13:50:35.139456 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 18693
2026-09-26T13:50:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3462 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:35 dist-test-slave-r34j krb5kdc[19329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:35 dist-test-slave-r34j krb5kdc[19329](info): set up 2 sockets
Sep 26 13:50:35 dist-test-slave-r34j krb5kdc[19329](info): commencing operation
krb5kdc: starting...
W20260926 13:50:36.216756 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:36 dist-test-slave-r34j krb5kdc[19329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:36.342671 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:46621
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:46621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:36.452627 19342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:36.452927 19342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:36.453013 19342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:36.457083 19342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:36.457171 19342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:36.457207 19342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:36.457267 19342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:36.457324 19342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:36.462774 19342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:46621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:46621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:36.463991 19342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:36.465070 19342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:36.473892 19350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:36.473886 19347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:36.473897 19348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:36.474238 19342 server_base.cc:1034] running on GCE node
I20260926 13:50:36.474840 19342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:36.475993 19342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:36.477177 19342 hybrid_clock.cc:648] HybridClock initialized: now 1790430636477171 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T13:50:36Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1349 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:36 dist-test-slave-r34j krb5kdc[19355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:36 dist-test-slave-r34j krb5kdc[19355](info): set up 2 sockets
Sep 26 13:50:36 dist-test-slave-r34j krb5kdc[19355](info): commencing operation
krb5kdc: starting...
W20260926 13:50:37.556326 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:37 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:37.684084 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45309
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:37.794389 19368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:37.794678 19368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:37.794720 19368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:37.798802 19368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:37.798866 19368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:37.798885 19368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:37.798904 19368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:37.798919 19368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:37.804236 19368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:45309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:37.805298 19368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:37.806283 19368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:37.815270 19376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:37.815258 19373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:37.815258 19374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:37.815433 19368 server_base.cc:1034] running on GCE node
I20260926 13:50:37.816234 19368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:37.817420 19368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:37.818622 19368 hybrid_clock.cc:648] HybridClock initialized: now 1790430637818596 us; error 43 us; skew 500 ppm
Sep 26 13:50:37 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430637, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:37.821863 19368 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:37.822774 19368 webserver.cc:460] Webserver started at http://127.10.236.254:36995/ using document root <none> and password file <none>
I20260926 13:50:37.823351 19368 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:37.823441 19368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:37.823695 19368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:37.825809 19368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "29356993df1541a3bfb202b5df0232cd"
format_stamp: "Formatted at 2026-09-26 13:50:37 on dist-test-slave-r34j"
server_key: "ff42c18fd080608918258055c816ff84"
server_key_iv: "cd3f72e4f04470d2e22752bf23ec164d"
server_key_version: "encryptionkey@0"
I20260926 13:50:37.826385 19368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "29356993df1541a3bfb202b5df0232cd"
format_stamp: "Formatted at 2026-09-26 13:50:37 on dist-test-slave-r34j"
server_key: "ff42c18fd080608918258055c816ff84"
server_key_iv: "cd3f72e4f04470d2e22752bf23ec164d"
server_key_version: "encryptionkey@0"
I20260926 13:50:37.830108 19368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260926 13:50:37.832665 19383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:37.833945 19368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 13:50:37.834105 19368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "29356993df1541a3bfb202b5df0232cd"
format_stamp: "Formatted at 2026-09-26 13:50:37 on dist-test-slave-r34j"
server_key: "ff42c18fd080608918258055c816ff84"
server_key_iv: "cd3f72e4f04470d2e22752bf23ec164d"
server_key_version: "encryptionkey@0"
I20260926 13:50:37.834220 19368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:37.869304 19368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:37.872813 19368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:37.873035 19368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:37.880373 19368 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:45309
I20260926 13:50:37.880429 19435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:45309 every 8 connection(s)
I20260926 13:50:37.881423 19368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:37.884383 19436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:37.889843 19436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Bootstrap starting.
I20260926 13:50:37.890969 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19368
I20260926 13:50:37.891135 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:37.891414 11187 external_mini_cluster.cc:1428] Setting key d568eba5faaa4aa3320faa7fe23cd5ae
I20260926 13:50:37.892649 19436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:37.893426 19436 log.cc:824] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:37.895736 19436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: No bootstrap required, opened a new log
I20260926 13:50:37.900247 19436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:37.900502 19436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:37.900590 19436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29356993df1541a3bfb202b5df0232cd, State: Initialized, Role: FOLLOWER
Sep 26 13:50:37 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:37.901103 19436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:37.901255 19436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:37.901355 19436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:37.901463 19436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:37.902698 19436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:37.903108 19436 leader_election.cc:302] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29356993df1541a3bfb202b5df0232cd; no voters:
I20260926 13:50:37.903466 19436 leader_election.cc:288] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:37.903743 19441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:37.903967 19441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 LEADER]: Becoming Leader. State: Replica: 29356993df1541a3bfb202b5df0232cd, State: Running, Role: LEADER
I20260926 13:50:37.904400 19441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:37.905181 19436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:37.906374 19442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29356993df1541a3bfb202b5df0232cd. Latest consensus state: current_term: 1 leader_uuid: "29356993df1541a3bfb202b5df0232cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } } }
I20260926 13:50:37.906899 19442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:37.906765 19443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29356993df1541a3bfb202b5df0232cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } } }
I20260926 13:50:37.907290 19443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:37.907725 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:37.892572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.10.236.254:45309)
0926 13:50:37.893000 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:37.893020 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:37.893055 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:50:37.893725 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:37.893742 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:37.894187 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:37.894499 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:37.895372 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:37.896473 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:37.897679 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:37.898165 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:37.898295 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:37.901390 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:37.901398 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:37.901417 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:37.901450 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:37.903663 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:37.904282 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:37.904288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:37.904294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:37.904433 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:37.904746 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:37.904749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:37.904750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:37.905453 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:37.905720 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:37.906340 (+ 620us) server_negotiation.cc:299] Negotiation successful
0926 13:50:37.906691 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":165,"threads_started":1}
I20260926 13:50:37.907968 19451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:37.910692 19451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:37.916917 19451 catalog_manager.cc:1347] Generated new cluster ID: dd970f0e431b4a6b9b38571fc3c33cb8
I20260926 13:50:37.917007 19451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:37.930845 19451 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:37.931845 19451 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:37.948884 19451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Generated new TSK 0
I20260926 13:50:37.949698 19451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:37.984087 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45309
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:38.096307 19462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:38.096560 19462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:38.096601 19462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:38.100728 19462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:38.100793 19462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:38.100888 19462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:38.106302 19462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:38.107436 19462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:38.108467 19462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:38.118048 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.118052 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.118108 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:38.118343 19462 server_base.cc:1034] running on GCE node
I20260926 13:50:38.118741 19462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:38.119398 19462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:38.120595 19462 hybrid_clock.cc:648] HybridClock initialized: now 1790430638120577 us; error 32 us; skew 500 ppm
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:38.123420 19462 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:38.124418 19462 webserver.cc:460] Webserver started at http://127.10.236.193:34177/ using document root <none> and password file <none>
I20260926 13:50:38.124981 19462 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:38.125078 19462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:38.125305 19462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:38.127329 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "24e61c78aef94c25b74797743c5f9397"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "96a229cc40fab19a63199f95371c83b0"
server_key_iv: "4c529f421a624af744c157fdf2d1e4e5"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.127947 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "24e61c78aef94c25b74797743c5f9397"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "96a229cc40fab19a63199f95371c83b0"
server_key_iv: "4c529f421a624af744c157fdf2d1e4e5"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.131724 19462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:50:38.134105 19477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.135311 19462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:38.135440 19462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "24e61c78aef94c25b74797743c5f9397"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "96a229cc40fab19a63199f95371c83b0"
server_key_iv: "4c529f421a624af744c157fdf2d1e4e5"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.135653 19462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:38.152004 19462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:38.155367 19462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:38.155642 19462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:38.156229 19462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:38.157065 19462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:38.157135 19462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.157207 19462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:38.157258 19462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.167475 19462 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:37895
I20260926 13:50:38.167517 19590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:37895 every 8 connection(s)
I20260926 13:50:38.168444 19462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:38.171113 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19462
I20260926 13:50:38.171259 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:38.171705 11187 external_mini_cluster.cc:1428] Setting key bc8803e66ad09bb04933b5bf1d36a99a
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:50:38.183962 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:38.170383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:52937 (local address 127.10.236.254:45309)
0926 13:50:38.170685 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:38.170689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:38.171672 (+ 983us) server_negotiation.cc:407] Connection header received
0926 13:50:38.172756 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:38.172759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:38.172803 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:38.172898 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:38.174224 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.175165 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.176586 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.176863 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.176988 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:38.179945 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:38.179949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:38.179951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:38.179984 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:38.181752 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:38.182327 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:38.182337 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:38.182339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:38.182414 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:38.182774 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:38.182777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:38.182779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:38.182967 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:38.183081 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:38.183640 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 13:50:38.183783 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260926 13:50:38.184806 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:38.170776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:52937)
0926 13:50:38.171500 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:38.171562 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:38.172548 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:38.173087 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:38.173096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:38.173501 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:38.173973 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:38.173985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.175307 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.175310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:38.176447 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:38.176457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.177028 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.177033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:38.177220 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:38.177874 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:38.177908 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:38.179760 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:38.181884 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:38.181890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:38.181908 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:38.182209 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:38.182521 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:38.182524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:38.182526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:38.182670 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:38.183080 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:38.183089 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:38.183349 (+ 260us) client_negotiation.cc:769] Sending connection context
0926 13:50:38.183589 (+ 240us) client_negotiation.cc:240] Negotiation successful
0926 13:50:38.183924 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":140,"threads_started":1}
I20260926 13:50:38.186095 19591 heartbeater.cc:347] Connected to a master server at 127.10.236.254:45309
I20260926 13:50:38.186357 19591 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:38.186861 19591 heartbeater.cc:511] Master 127.10.236.254:45309 requested a full tablet report, sending...
I20260926 13:50:38.188373 19400 ts_manager.cc:194] Registered new tserver with Master: 24e61c78aef94c25b74797743c5f9397 (127.10.236.193:37895)
I20260926 13:50:38.189484 19400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:52937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:38.199031 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45309
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:38.310032 19596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:38.310329 19596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:38.310375 19596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:38.314514 19596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:38.314568 19596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:38.314625 19596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:38.319746 19596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:38.320721 19596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:38.321661 19596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:38.331262 19596 server_base.cc:1034] running on GCE node
W20260926 13:50:38.331234 19601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.331197 19604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.331190 19602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:38.331887 19596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:38.332543 19596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:38.333705 19596 hybrid_clock.cc:648] HybridClock initialized: now 1790430638333690 us; error 28 us; skew 500 ppm
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:38.336535 19596 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:38.337428 19596 webserver.cc:460] Webserver started at http://127.10.236.194:45213/ using document root <none> and password file <none>
I20260926 13:50:38.338052 19596 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:38.338138 19596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:38.338366 19596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:38.340623 19596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "132be62bacf144e7b4a356b5c2d2e2f0"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "e0b401331adf4819cbf9e6596b61bb4f"
server_key_iv: "17949babada41bbd0051423ccb53cfe4"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.341173 19596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "132be62bacf144e7b4a356b5c2d2e2f0"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "e0b401331adf4819cbf9e6596b61bb4f"
server_key_iv: "17949babada41bbd0051423ccb53cfe4"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.345005 19596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:38.347688 19611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.349151 19596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:38.349314 19596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "132be62bacf144e7b4a356b5c2d2e2f0"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "e0b401331adf4819cbf9e6596b61bb4f"
server_key_iv: "17949babada41bbd0051423ccb53cfe4"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.349452 19596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:38.373294 19596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:38.376994 19596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:38.377224 19596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:38.377868 19596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:38.378767 19596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:38.378842 19596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.378912 19596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:38.378964 19596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.388331 19596 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:43283
I20260926 13:50:38.388414 19724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:43283 every 8 connection(s)
I20260926 13:50:38.389310 19596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:38.395782 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19596
I20260926 13:50:38.395932 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:38.396204 11187 external_mini_cluster.cc:1428] Setting key ca9e2b1930f56233e1d3cc73414b9165
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:38.403692 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:38.391217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:33931 (local address 127.10.236.254:45309)
0926 13:50:38.391356 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:38.391359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:38.392459 (+ 1100us) server_negotiation.cc:407] Connection header received
0926 13:50:38.393343 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:38.393347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:38.393403 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:38.393528 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:38.394683 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.395216 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.396602 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.396830 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.396943 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:38.399691 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:38.399698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:38.399704 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:38.399738 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:38.401423 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:38.401986 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:38.401994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:38.401997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:38.402093 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:38.402425 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:38.402430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:38.402433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:38.402694 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:38.402809 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:38.403296 (+ 487us) server_negotiation.cc:299] Negotiation successful
0926 13:50:38.403435 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:50:38.404433 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:38.391665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:33931)
0926 13:50:38.392301 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:38.392339 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:38.393140 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:38.393696 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:38.393703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:38.394061 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:38.394539 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:38.394553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.395346 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.395351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:38.396480 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:38.396489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.396978 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.396983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:38.397156 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:38.397761 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:38.397777 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:38.399463 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:38.401540 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:38.401546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:38.401560 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:38.401841 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:38.402191 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:38.402193 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:38.402195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:38.402321 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:38.402798 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:38.402803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:38.403029 (+ 226us) client_negotiation.cc:769] Sending connection context
0926 13:50:38.403193 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 13:50:38.403440 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":117,"threads_started":1}
I20260926 13:50:38.405635 19725 heartbeater.cc:347] Connected to a master server at 127.10.236.254:45309
I20260926 13:50:38.405928 19725 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:38.406520 19725 heartbeater.cc:511] Master 127.10.236.254:45309 requested a full tablet report, sending...
I20260926 13:50:38.407938 19400 ts_manager.cc:194] Registered new tserver with Master: 132be62bacf144e7b4a356b5c2d2e2f0 (127.10.236.194:43283)
I20260926 13:50:38.408595 19400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:33931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:38.427201 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45309
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:38.538615 19730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:38.538926 19730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:38.539018 19730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:38.543212 19730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:38.543329 19730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:38.543435 19730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:38.548977 19730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:38.550105 19730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:38.551157 19730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:38.561165 19738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.561205 19735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.561174 19736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:38.561542 19730 server_base.cc:1034] running on GCE node
I20260926 13:50:38.562096 19730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:38.562698 19730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:38.563876 19730 hybrid_clock.cc:648] HybridClock initialized: now 1790430638563827 us; error 54 us; skew 500 ppm
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:38.567001 19730 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:38.568037 19730 webserver.cc:460] Webserver started at http://127.10.236.195:35709/ using document root <none> and password file <none>
I20260926 13:50:38.568624 19730 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:38.568686 19730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:38.568900 19730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:38.570915 19730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "9e7423a017004db7835d46749f0292a7"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "ccf27d8548595a3afc439107416221ec"
server_key_iv: "2aacd759d6cc83dc4b43300c3a9ce4c3"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.571473 19730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "9e7423a017004db7835d46749f0292a7"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "ccf27d8548595a3afc439107416221ec"
server_key_iv: "2aacd759d6cc83dc4b43300c3a9ce4c3"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.575433 19730 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:50:38.578063 19745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.579314 19730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:38.579470 19730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "9e7423a017004db7835d46749f0292a7"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "ccf27d8548595a3afc439107416221ec"
server_key_iv: "2aacd759d6cc83dc4b43300c3a9ce4c3"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.579793 19730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:38.603258 19730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:38.607049 19730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:38.607270 19730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:38.607982 19730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:38.608843 19730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:38.608915 19730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.608980 19730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:38.609032 19730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.619417 19730 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:44351
I20260926 13:50:38.619454 19858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:44351 every 8 connection(s)
I20260926 13:50:38.620601 19730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:38.624065 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19730
I20260926 13:50:38.624222 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:38.624492 11187 external_mini_cluster.cc:1428] Setting key e6d857af62737010d669bb2d6b480bc6
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:38.634764 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:38.622479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:53463 (local address 127.10.236.254:45309)
0926 13:50:38.622651 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:38.622655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:38.623834 (+ 1179us) server_negotiation.cc:407] Connection header received
0926 13:50:38.624895 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:38.624898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:38.624968 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:38.625066 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:38.626784 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.627361 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.628617 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.628845 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.628938 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:38.631547 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:38.631552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:38.631555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:38.631587 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:38.632946 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:38.633420 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:38.633423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:38.633424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:38.633570 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:38.633874 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:38.633876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:38.633878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:38.634045 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:38.634121 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:38.634557 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 13:50:38.634645 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 13:50:38.635648 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:38.622877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:53463)
0926 13:50:38.623638 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:38.623681 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:38.624672 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:38.625251 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:38.625262 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:38.625937 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:38.626629 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:38.626651 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.627546 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.627551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:38.628485 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:38.628494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:38.629005 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:38.629008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:38.629161 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:38.629821 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:38.629837 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:38.631273 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:38.633068 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:38.633073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:38.633084 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:38.633324 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:38.633667 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:38.633669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:38.633670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:38.633792 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:38.634132 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:38.634137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:38.634350 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 13:50:38.634501 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 13:50:38.634744 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":116,"threads_started":1}
I20260926 13:50:38.636819 19859 heartbeater.cc:347] Connected to a master server at 127.10.236.254:45309
I20260926 13:50:38.637053 19859 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:38.637671 19859 heartbeater.cc:511] Master 127.10.236.254:45309 requested a full tablet report, sending...
I20260926 13:50:38.638795 19400 ts_manager.cc:194] Registered new tserver with Master: 9e7423a017004db7835d46749f0292a7 (127.10.236.195:44351)
I20260926 13:50:38.639364 19400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:53463
I20260926 13:50:38.650549 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab.
Entry for principal oryx/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab.
WARNING: no policy specified for oryx/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab.
Entry for principal oryx/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab.
WARNING: no policy specified for oryx/127.10.236.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab.
Entry for principal oryx/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab.
I20260926 13:50:38.730754 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19462
I20260926 13:50:38.736331 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19596
I20260926 13:50:38.741483 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19730
I20260926 13:50:38.746605 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19368
I20260926 13:50:38.752830 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45309
--webserver_interface=127.10.236.254
--webserver_port=36995
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:38.863377 19868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:38.863684 19868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:38.863725 19868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:38.868139 19868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:38.868206 19868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:38.868225 19868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:38.868242 19868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:38.868257 19868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:38.873683 19868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:45309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=36995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:38.874832 19868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:38.875931 19868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:38.885186 19876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.885253 19874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:38.885254 19873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:38.885360 19868 server_base.cc:1034] running on GCE node
I20260926 13:50:38.886205 19868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:38.887410 19868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:38.888638 19868 hybrid_clock.cc:648] HybridClock initialized: now 1790430638888600 us; error 50 us; skew 500 ppm
Sep 26 13:50:38 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430638, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:38.891724 19868 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:38.892683 19868 webserver.cc:460] Webserver started at http://127.10.236.254:36995/ using document root <none> and password file <none>
I20260926 13:50:38.893241 19868 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:38.893321 19868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:38.897583 19868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:50:38.899868 19883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:38.901186 19868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:38.901290 19868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "29356993df1541a3bfb202b5df0232cd"
format_stamp: "Formatted at 2026-09-26 13:50:37 on dist-test-slave-r34j"
server_key: "ff42c18fd080608918258055c816ff84"
server_key_iv: "cd3f72e4f04470d2e22752bf23ec164d"
server_key_version: "encryptionkey@0"
I20260926 13:50:38.901826 19868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:38.932574 19868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:38.936028 19868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:38.936201 19868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:38.943305 19868 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:45309
I20260926 13:50:38.943317 19935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:45309 every 8 connection(s)
I20260926 13:50:38.944414 19868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:38.948968 19936 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:50:38.949998 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19868
I20260926 13:50:38.950501 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:37895
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:45309
--webserver_port=34177
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:50:38.950943 19936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Bootstrap starting.
I20260926 13:50:38.968997 19936 log.cc:824] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:38.975709 19936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:50:38.976104 19936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Bootstrap complete.
I20260926 13:50:38.980327 19936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:38.980765 19936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29356993df1541a3bfb202b5df0232cd, State: Initialized, Role: FOLLOWER
I20260926 13:50:38.981259 19936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:38.981408 19936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:38.981504 19936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:38.981626 19936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:50:38.983754 19936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:38.984107 19936 leader_election.cc:302] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29356993df1541a3bfb202b5df0232cd; no voters:
I20260926 13:50:38.984496 19936 leader_election.cc:288] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:50:38.984580 19940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:50:38.984827 19940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [term 2 LEADER]: Becoming Leader. State: Replica: 29356993df1541a3bfb202b5df0232cd, State: Running, Role: LEADER
I20260926 13:50:38.985199 19940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } }
I20260926 13:50:38.985786 19936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:38.986732 19942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29356993df1541a3bfb202b5df0232cd. Latest consensus state: current_term: 2 leader_uuid: "29356993df1541a3bfb202b5df0232cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } } }
I20260926 13:50:38.986881 19942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:38.987150 19941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "29356993df1541a3bfb202b5df0232cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29356993df1541a3bfb202b5df0232cd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45309 } } }
I20260926 13:50:38.987229 19941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:38.987967 19948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:38.990649 19948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:38.991232 19948 catalog_manager.cc:1259] Loaded cluster ID: dd970f0e431b4a6b9b38571fc3c33cb8
I20260926 13:50:38.991318 19948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:38.993067 19948 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:38.993634 19948 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 29356993df1541a3bfb202b5df0232cd: Loaded TSK: 0
I20260926 13:50:38.994599 19948 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:50:39.079584 19938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:39.079862 19938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:39.079908 19938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:39.084035 19938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:39.084120 19938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:39.084218 19938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:39.089718 19938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.193.keytab
--principal=oryx/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:37895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=34177
--tserver_master_addrs=127.10.236.254:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:39.090705 19938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:39.091843 19938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:39.102869 19962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:39.102984 19965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:39.102869 19963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:39.103266 19938 server_base.cc:1034] running on GCE node
I20260926 13:50:39.103993 19938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:39.104766 19938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:39.106033 19938 hybrid_clock.cc:648] HybridClock initialized: now 1790430639106018 us; error 29 us; skew 500 ppm
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430639, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:39.109539 19938 init.cc:377] Logged in from keytab as oryx/127.10.236.193@KRBTEST.COM (short username oryx)
I20260926 13:50:39.110644 19938 webserver.cc:460] Webserver started at http://127.10.236.193:34177/ using document root <none> and password file <none>
I20260926 13:50:39.111326 19938 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:39.111454 19938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:39.116530 19938 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:50:39.119097 19972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.120715 19938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:39.120846 19938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "24e61c78aef94c25b74797743c5f9397"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "96a229cc40fab19a63199f95371c83b0"
server_key_iv: "4c529f421a624af744c157fdf2d1e4e5"
server_key_version: "encryptionkey@0"
I20260926 13:50:39.121608 19938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:39.143720 19938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:39.147225 19938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:39.147535 19938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:39.148155 19938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:39.149010 19938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:39.149086 19938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.149159 19938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:39.149204 19938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.161401 19938 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:37895
I20260926 13:50:39.161439 20085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:37895 every 8 connection(s)
I20260926 13:50:39.162846 19938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:39.169718 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 19938
I20260926 13:50:39.170228 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:43283
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:45309
--webserver_port=45213
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.179026 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.165765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:45213)
0926 13:50:39.166609 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.166672 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.167770 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.168768 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.168775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.169144 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.169613 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.169626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.171612 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.171618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.173293 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.173304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.174435 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.174439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.174597 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.175342 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.175359 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.178203 (+ 2844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":126,"threads_started":1}
W20260926 13:50:39.179534 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.167205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:45213 (local address 127.10.236.254:45309)
0926 13:50:39.167846 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.167860 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.167905 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:50:39.168103 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.168113 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.168395 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.168660 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.169807 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.171370 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.173430 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.174245 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.174439 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.178509 (+ 4070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:45213: BlockingRecv error: recv got EOF from 127.10.236.193:45213 (error 108)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":95,"threads_started":1}
W20260926 13:50:39.179675 20086 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:45309 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.236.254:45309: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.185985 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.180154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:46899)
0926 13:50:39.180345 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.180364 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.180455 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.180840 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.180843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.181101 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.181197 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.181203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.182013 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.182020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.182619 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.182627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.183111 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.183115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.183210 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.183761 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.183773 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.185798 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260926 13:50:39.186033 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.180254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:46899 (local address 127.10.236.254:45309)
0926 13:50:39.180561 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.180567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.180583 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:39.180629 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.180631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.180705 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.180812 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.181322 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.181886 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.182745 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.182993 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.183111 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.185918 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:46899: BlockingRecv error: recv got EOF from 127.10.236.193:46899 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.192647 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.186632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:49933)
0926 13:50:39.186775 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.186792 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.186897 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.187152 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.187155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.187314 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.187400 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.187406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.188461 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.188465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.189365 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.189374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.189828 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.189831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.189912 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.190383 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.190397 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.192514 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 13:50:39.192950 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.186663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:49933 (local address 127.10.236.254:45309)
0926 13:50:39.186778 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.186783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.186856 (+ 73us) server_negotiation.cc:407] Connection header received
0926 13:50:39.187006 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.187008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.187050 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.187123 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.187583 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.188337 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.189509 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.189726 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.189808 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.192676 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:49933: BlockingRecv error: recv got EOF from 127.10.236.193:49933 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 13:50:39.193070 20086 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:50:39.286209 20090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:39.286562 20090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:39.286669 20090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:39.291014 20090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:39.291114 20090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:39.291216 20090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:39.296607 20090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.194.keytab
--principal=oryx/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:43283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=45213
--tserver_master_addrs=127.10.236.254:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:39.297746 20090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:39.298844 20090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:39.308818 20096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:39.308800 20095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:39.308800 20098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:39.308976 20090 server_base.cc:1034] running on GCE node
I20260926 13:50:39.309572 20090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:39.310269 20090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:39.311458 20090 hybrid_clock.cc:648] HybridClock initialized: now 1790430639311434 us; error 38 us; skew 500 ppm
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430639, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:39.314316 20090 init.cc:377] Logged in from keytab as oryx/127.10.236.194@KRBTEST.COM (short username oryx)
I20260926 13:50:39.315197 20090 webserver.cc:460] Webserver started at http://127.10.236.194:45213/ using document root <none> and password file <none>
I20260926 13:50:39.315819 20090 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:39.315927 20090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:39.319757 20090 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260926 13:50:39.321894 20105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.323174 20090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:39.323307 20090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "132be62bacf144e7b4a356b5c2d2e2f0"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "e0b401331adf4819cbf9e6596b61bb4f"
server_key_iv: "17949babada41bbd0051423ccb53cfe4"
server_key_version: "encryptionkey@0"
I20260926 13:50:39.323918 20090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:39.348730 20090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:39.352421 20090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:39.352632 20090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:39.353221 20090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:39.354054 20090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:39.354123 20090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.354187 20090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:39.354240 20090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.363372 20090 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:43283
I20260926 13:50:39.363408 20218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:43283 every 8 connection(s)
I20260926 13:50:39.364410 20090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:39.368052 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20090
I20260926 13:50:39.368506 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:44351
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:45309
--webserver_port=35709
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:50:39.377544 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.366477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:50761 (local address 127.10.236.254:45309)
0926 13:50:39.366633 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.366637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.367465 (+ 828us) server_negotiation.cc:407] Connection header received
0926 13:50:39.368835 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.368838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.368904 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.369006 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.370466 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.370994 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.372848 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.373064 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.373154 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.377439 (+ 4285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:50761: BlockingRecv error: recv got EOF from 127.10.236.194:50761 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:39.378504 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.366858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:50761)
0926 13:50:39.367318 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.367358 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.368591 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.369201 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.369208 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.369551 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.370259 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.370281 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.371116 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.371123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.372708 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.372724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.373242 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.373249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.373489 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.374150 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.374177 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.377322 (+ 3145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":116,"threads_started":1}
W20260926 13:50:39.378997 20219 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:45309 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.236.254:45309: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.385282 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.379565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:54557)
0926 13:50:39.379691 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.379709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.379803 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.380075 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.380080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.380277 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.380413 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.380423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.381178 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.381181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.382081 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.382091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.382564 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.382567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.382660 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.383140 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.383154 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.385132 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 13:50:39.385417 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.379658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:54557 (local address 127.10.236.254:45309)
0926 13:50:39.379758 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.379764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.379782 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:50:39.379917 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.379919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.379960 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.380020 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.380566 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.381070 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.382198 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.382448 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.382530 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.385269 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:54557: BlockingRecv error: recv got EOF from 127.10.236.194:54557 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.391326 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.385989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:36597)
0926 13:50:39.386109 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.386123 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.386194 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.386408 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.386411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.386578 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.386715 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.386721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.387601 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.387604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.388471 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.388480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.388870 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.388873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.388954 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.389389 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.389402 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.391189 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 13:50:39.391568 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.386090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:36597 (local address 127.10.236.254:45309)
0926 13:50:39.386191 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.386194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.386206 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:50:39.386274 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.386276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.386310 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.386408 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.386818 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.387405 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.388573 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.388772 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.388850 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.391334 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:36597: BlockingRecv error: recv got EOF from 127.10.236.194:36597 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 13:50:39.391702 20219 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:50:39.484021 20222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:39.484351 20222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:39.484441 20222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:39.488932 20222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:39.489015 20222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:39.489110 20222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:39.494544 20222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/krb5kdc/oryx_127.10.236.195.keytab
--principal=oryx/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=35709
--tserver_master_addrs=127.10.236.254:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:39.495822 20222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:39.496843 20222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:39.507094 20230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:39.507262 20222 server_base.cc:1034] running on GCE node
W20260926 13:50:39.507104 20227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:39.507218 20228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:39.507814 20222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:39.508402 20222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:39.509581 20222 hybrid_clock.cc:648] HybridClock initialized: now 1790430639509559 us; error 31 us; skew 500 ppm
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430639, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:39.512599 20222 init.cc:377] Logged in from keytab as oryx/127.10.236.195@KRBTEST.COM (short username oryx)
I20260926 13:50:39.513535 20222 webserver.cc:460] Webserver started at http://127.10.236.195:35709/ using document root <none> and password file <none>
I20260926 13:50:39.514124 20222 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:39.514241 20222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:39.518285 20222 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:39.520519 20237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.521771 20222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:39.521909 20222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "9e7423a017004db7835d46749f0292a7"
format_stamp: "Formatted at 2026-09-26 13:50:38 on dist-test-slave-r34j"
server_key: "ccf27d8548595a3afc439107416221ec"
server_key_iv: "2aacd759d6cc83dc4b43300c3a9ce4c3"
server_key_version: "encryptionkey@0"
I20260926 13:50:39.522456 20222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:39.551748 20222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:39.554909 20222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:39.555123 20222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:39.555742 20222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:39.556551 20222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:39.556622 20222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.556694 20222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:39.556736 20222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:39.565527 20222 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:44351
I20260926 13:50:39.565547 20350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:44351 every 8 connection(s)
I20260926 13:50:39.566573 20222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.576145 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20222
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:50:39.577729 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.568432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:36951 (local address 127.10.236.254:45309)
0926 13:50:39.568616 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.568619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.569671 (+ 1052us) server_negotiation.cc:407] Connection header received
0926 13:50:39.570524 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.570527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.570573 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.570647 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.571934 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.572444 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.573611 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.573807 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.573879 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.577586 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:36951: BlockingRecv error: recv got EOF from 127.10.236.195:36951 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 13:50:39.578439 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.568814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:36951)
0926 13:50:39.569525 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.569565 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.570324 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.570811 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.570818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.571194 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.571807 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.571822 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.572587 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.572590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.573506 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.573515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.573925 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.573928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.574083 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.574681 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.574697 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.577464 (+ 2767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":105,"threads_started":1}
W20260926 13:50:39.578896 20351 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:45309 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.236.254:45309: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.585665 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.579279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:51753)
0926 13:50:39.580276 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.580290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.580390 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.580676 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.580679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.580838 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.580947 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.580956 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.581762 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.581768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.582493 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.582501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.583094 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.583097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.583186 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.583636 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.583647 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.585552 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 13:50:39.585795 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.579367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:51753 (local address 127.10.236.254:45309)
0926 13:50:39.579473 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.579476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.580335 (+ 859us) server_negotiation.cc:407] Connection header received
0926 13:50:39.580478 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.580481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.580523 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.580606 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.581079 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.581654 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.582648 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.582979 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.583097 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.585658 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:51753: BlockingRecv error: recv got EOF from 127.10.236.195:51753 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:50:39.587146 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.577203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.10.236.254:45309)
0926 13:50:39.577468 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.577472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.577483 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:50:39.577517 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.577522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.577580 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.577663 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.578562 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.579323 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.580223 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.580546 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.580679 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.582264 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:39.582269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:39.582283 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:39.582307 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:39.584216 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:39.584685 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:39.584695 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:39.584705 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:39.584865 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:39.585176 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:39.585180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:39.585183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:39.585675 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:39.585859 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:39.586713 (+ 854us) server_negotiation.cc:299] Negotiation successful
0926 13:50:39.586947 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:39 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:39.591509 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:39.586207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:42015)
0926 13:50:39.586348 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:39.586361 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:39.586431 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:39.586751 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:39.586753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:39.586904 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:39.586986 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.586992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.587991 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.587994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.588595 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:39.588604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.589102 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.589106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:39.589192 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.589683 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:39.589696 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:39.591361 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260926 13:50:39.591602 20089 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:39.586269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:42015 (local address 127.10.236.254:45309)
0926 13:50:39.586404 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:39.586409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:39.586421 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:50:39.586580 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:39.586583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:39.586623 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:39.586719 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:39.587125 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.587870 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.588711 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:39.588984 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:39.589101 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:39.591472 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:42015: BlockingRecv error: recv got EOF from 127.10.236.195:42015 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 13:50:39.591742 20351 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 13:50:40 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:40 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:40.199846 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:40.193700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:36477)
0926 13:50:40.193971 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:40.193989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:40.194111 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:40.194379 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:40.194381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:40.194572 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:40.194689 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:40.194694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.196026 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.196031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:40.196779 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:40.196789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.197218 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.197220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:40.197298 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:40.197842 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:40.197859 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:40.199729 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":68,"threads_started":1}
W20260926 13:50:40.200189 20357 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:40.193854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:36477 (local address 127.10.236.254:45309)
0926 13:50:40.194110 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:40.194114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:40.194127 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:50:40.194186 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:40.194188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:40.194229 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:40.194344 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:40.194936 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.195861 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.196886 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.197116 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.197198 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:40.200022 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:36477: BlockingRecv error: recv got EOF from 127.10.236.193:36477 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
Sep 26 13:50:40 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:40 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:40.398476 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:40.392426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:39221)
0926 13:50:40.392793 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:40.392815 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:40.392969 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:40.393268 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:40.393270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:40.393480 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:40.393599 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:40.393606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.394387 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.394393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:40.395113 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:40.395122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.395612 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.395615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:40.395720 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:40.396303 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:40.396317 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:40.398361 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":89,"threads_started":1}
W20260926 13:50:40.398650 20357 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:40.392572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:39221 (local address 127.10.236.254:45309)
0926 13:50:40.392751 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:40.392756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:40.392895 (+ 139us) server_negotiation.cc:407] Connection header received
0926 13:50:40.393094 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:40.393097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:40.393150 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:40.393241 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:40.393747 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.394273 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.395229 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.395470 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.395613 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:40.398501 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:39221: BlockingRecv error: recv got EOF from 127.10.236.194:39221 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 26 13:50:40 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:40 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:40.597738 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:40.592339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:59885)
0926 13:50:40.592602 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:40.592619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:40.592727 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:40.592956 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:40.592959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:40.593140 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:40.593240 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:40.593246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.593956 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.593959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:40.594534 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:40.594543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.594995 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.594998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:40.595067 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:40.595699 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:40.595712 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:40.597617 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
W20260926 13:50:40.597895 20357 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:40.592418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:59885 (local address 127.10.236.254:45309)
0926 13:50:40.592549 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:40.592552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:40.592695 (+ 143us) server_negotiation.cc:407] Connection header received
0926 13:50:40.592803 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:40.592805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:40.592849 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:40.592916 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:40.593335 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.593833 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.594657 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:40.594868 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:40.594945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:40.597739 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:59885: BlockingRecv error: recv got EOF from 127.10.236.195:59885 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 13:50:41 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:41 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:41.206604 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:41.200706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:35273)
0926 13:50:41.200971 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:41.200985 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:41.201072 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:41.201318 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:41.201321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:41.201493 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:41.201584 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:41.201591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.202402 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.202404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:41.202959 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:41.202967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.203625 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.203630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:41.203726 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:41.204380 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:41.204399 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:41.206454 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":95,"threads_started":1}
W20260926 13:50:41.206770 20361 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:41.200837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:35273 (local address 127.10.236.254:45309)
0926 13:50:41.201101 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:41.201105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:41.201118 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:50:41.201156 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:41.201158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:41.201199 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:41.201290 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:41.201712 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.202305 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.203077 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.203431 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.203604 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:41.206642 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:35273: BlockingRecv error: recv got EOF from 127.10.236.193:35273 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
Sep 26 13:50:41 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:41 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:41.405058 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:41.399321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:41357)
0926 13:50:41.399616 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:41.399633 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:41.399745 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:41.400006 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:41.400009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:41.400213 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:41.400301 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:41.400306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.401146 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.401150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:41.402021 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:41.402032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.402519 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.402521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:41.402582 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:41.403045 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:41.403057 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:41.404888 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
W20260926 13:50:41.405114 20361 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:41.399591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:41357 (local address 127.10.236.254:45309)
0926 13:50:41.399717 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:41.399721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:41.399736 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:41.399834 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:41.399838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:41.399890 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:41.399980 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:41.400419 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.401005 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.402171 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.402402 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.402482 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:41.405005 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:41357: BlockingRecv error: recv got EOF from 127.10.236.194:41357 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 13:50:41 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:41 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:41.608925 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:41.598617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:59347)
0926 13:50:41.599016 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:41.599035 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:41.599155 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:41.599443 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:41.599446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:41.599672 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:41.599760 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:41.599766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.600514 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.600520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:41.601073 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:41.601080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.601561 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.601564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:41.601651 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:41.606477 (+ 4826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:41.606491 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:41.608724 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":70,"threads_started":1}
W20260926 13:50:41.608995 20361 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:41.598971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:59347 (local address 127.10.236.254:45309)
0926 13:50:41.599115 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:41.599120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:41.599134 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:41.599275 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:41.599278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:41.599329 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:41.599420 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:41.599885 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.600403 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.601206 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:41.601427 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:41.601557 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:41.608869 (+ 7312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:59347: BlockingRecv error: recv got EOF from 127.10.236.195:59347 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 13:50:42 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:42 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:42.214145 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:42.207608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:44767)
0926 13:50:42.207906 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:42.207920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:42.208015 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:42.208329 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:42.208332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:42.208529 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:42.208627 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:42.208633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.209601 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.209606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:42.210490 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:42.210501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.210977 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.210980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:42.211046 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:42.211695 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:42.211708 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:42.213944 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":72,"threads_started":1}
W20260926 13:50:42.214258 20365 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:42.207683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44767 (local address 127.10.236.254:45309)
0926 13:50:42.208005 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:42.208009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:42.208025 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:42.208125 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:42.208128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:42.208170 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:42.208253 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:42.208765 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.209465 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.210617 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.210834 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.210913 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:42.214081 (+ 3168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:44767: BlockingRecv error: recv got EOF from 127.10.236.193:44767 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
Sep 26 13:50:42 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:42 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:42.412031 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:42.405911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:34703)
0926 13:50:42.406256 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:42.406269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:42.406373 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:42.406794 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:42.406797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:42.406973 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:42.407084 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:42.407090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.407963 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.407966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:42.408581 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:42.408591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.409042 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.409044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:42.409118 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:42.409696 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:42.409708 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:42.411857 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":113,"threads_started":1}
W20260926 13:50:42.412213 20365 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:42.405990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:34703 (local address 127.10.236.254:45309)
0926 13:50:42.406155 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:42.406160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:42.406365 (+ 205us) server_negotiation.cc:407] Connection header received
0926 13:50:42.406600 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:42.406603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:42.406654 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:42.406736 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:42.407221 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.407847 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.408714 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.408930 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.409022 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:42.412099 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:34703: BlockingRecv error: recv got EOF from 127.10.236.194:34703 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 13:50:42 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:42 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:42.615377 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:42.609768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:53059)
0926 13:50:42.610069 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:42.610083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:42.610189 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:42.610458 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:42.610461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:42.610658 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:42.610750 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:42.610756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.611619 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.611622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:42.612167 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:42.612175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.612650 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.612653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:42.612710 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:42.613300 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:42.613311 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:42.615213 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":138,"threads_started":1}
W20260926 13:50:42.615545 20365 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:42.609850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:53059 (local address 127.10.236.254:45309)
0926 13:50:42.610034 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:42.610042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:42.610186 (+ 144us) server_negotiation.cc:407] Connection header received
0926 13:50:42.610269 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:42.610272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:42.610322 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:42.610390 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:42.610888 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.611464 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.612291 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:42.612518 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:42.612610 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:42.615371 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:53059: BlockingRecv error: recv got EOF from 127.10.236.195:53059 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 13:50:43 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:43 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:43.221092 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:43.215127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:57437)
0926 13:50:43.215441 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:43.215456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:43.215608 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:43.215940 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:43.215943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:43.216150 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:43.216249 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:43.216254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.217131 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.217134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:43.217760 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:43.217769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.218289 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.218292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:43.218384 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:43.218957 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:43.218970 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:43.220911 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":89,"threads_started":1}
W20260926 13:50:43.221154 20369 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:43.215203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57437 (local address 127.10.236.254:45309)
0926 13:50:43.215525 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:43.215531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:43.215551 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:50:43.215759 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:43.215762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:43.215810 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:43.215893 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:43.216403 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.217004 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.217907 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.218140 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.218284 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:43.221037 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:57437: BlockingRecv error: recv got EOF from 127.10.236.193:57437 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":78,"threads_started":1}
Sep 26 13:50:43 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:43 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:43.418700 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:43.412783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:43807)
0926 13:50:43.413111 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:43.413147 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:43.413287 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:43.413560 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:43.413562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:43.413775 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:43.413864 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:43.413869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.414624 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.414627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:43.415205 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:43.415212 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.415735 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.415738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:43.415824 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:43.416362 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:43.416375 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:43.418529 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
W20260926 13:50:43.418861 20369 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:43.412907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:43807 (local address 127.10.236.254:45309)
0926 13:50:43.413080 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:43.413085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:43.413227 (+ 142us) server_negotiation.cc:407] Connection header received
0926 13:50:43.413393 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:43.413396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:43.413450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:43.413541 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:43.413985 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.414512 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.415328 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.415616 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.415732 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:43.418750 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:43807: BlockingRecv error: recv got EOF from 127.10.236.194:43807 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 13:50:43 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:43 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:43.622303 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:43.616274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:48913)
0926 13:50:43.616542 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:43.616557 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:43.616667 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:43.616972 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:43.616974 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:43.617151 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:43.617240 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:43.617245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.618041 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.618045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:43.618643 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:43.618651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.619211 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.619214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:43.619309 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:43.619969 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:43.619981 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:43.622187 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
W20260926 13:50:43.622462 20369 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:43.616428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:48913 (local address 127.10.236.254:45309)
0926 13:50:43.616616 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:43.616620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:43.616636 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:43.616813 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:43.616818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:43.616866 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:43.616944 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:43.617338 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.617893 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.618747 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:43.619107 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:43.619211 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:43.622334 (+ 3123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:48913: BlockingRecv error: recv got EOF from 127.10.236.195:48913 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 26 13:50:44 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:44 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:44.227952 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:44.222072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:42757)
0926 13:50:44.222355 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:44.222369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:44.222455 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:44.222721 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:44.222723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:44.222902 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:44.222991 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:44.222996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.223854 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.223857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:44.224466 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:44.224475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.224909 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.224911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:44.224972 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:44.225541 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:44.225561 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:44.227830 (+ 2269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":99,"threads_started":1}
W20260926 13:50:44.228154 20373 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:44.222175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:42757 (local address 127.10.236.254:45309)
0926 13:50:44.222414 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:44.222418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:44.222433 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:50:44.222563 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:44.222566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:44.222608 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:44.222686 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:44.223125 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.223725 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.224581 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.224809 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.224889 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:44.228000 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:42757: BlockingRecv error: recv got EOF from 127.10.236.193:42757 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":59,"threads_started":1}
Sep 26 13:50:44 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:44 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:44.424908 20374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:44.419581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:44359)
0926 13:50:44.419859 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:44.419872 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:44.419956 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:44.420241 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:44.420243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:44.420421 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:44.420508 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:44.420513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.421309 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.421312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:44.421914 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:44.421923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.422400 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.422402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:44.422460 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:44.422930 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:44.422941 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:44.424794 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":111,"threads_started":1}
W20260926 13:50:44.425104 20373 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:44.419743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:44359 (local address 127.10.236.254:45309)
0926 13:50:44.419888 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:44.419892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:44.419918 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:50:44.420075 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:44.420078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:44.420124 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:44.420199 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:44.420646 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.421200 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.422080 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.422298 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.422385 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:44.424936 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:44359: BlockingRecv error: recv got EOF from 127.10.236.194:44359 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 13:50:44 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:44 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:44.629647 20375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:44.623331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:35801)
0926 13:50:44.623721 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:44.623742 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:44.623886 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:44.624160 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:44.624163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:44.624416 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:44.624503 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:44.624508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.625294 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.625300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:44.625859 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:44.625866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.626383 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.626386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:44.626470 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:44.627096 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:44.627110 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:44.629477 (+ 2367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":117,"threads_started":1}
W20260926 13:50:44.629739 20373 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:44.623584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:35801 (local address 127.10.236.254:45309)
0926 13:50:44.623721 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:44.623727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:44.623826 (+ 99us) server_negotiation.cc:407] Connection header received
0926 13:50:44.623985 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:44.623989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:44.624047 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:44.624141 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:44.624621 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.625181 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.625975 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:44.626264 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:44.626382 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:44.629615 (+ 3233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:35801: BlockingRecv error: recv got EOF from 127.10.236.195:35801 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 13:50:45 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:45 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:45.234660 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:45.228742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:33041)
0926 13:50:45.229043 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:45.229065 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:45.229159 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:45.229386 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:45.229388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:45.229571 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:45.229666 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:45.229671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.230465 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.230469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:45.231221 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:45.231232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.231750 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.231753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:45.231828 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:45.232317 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:45.232329 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:45.234504 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":81,"threads_started":1}
W20260926 13:50:45.234858 20377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:45.228860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:33041 (local address 127.10.236.254:45309)
0926 13:50:45.229092 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:45.229095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:45.229111 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:45.229239 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:45.229242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:45.229282 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:45.229347 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:45.229785 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.230320 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.231358 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.231635 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.231724 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:45.234697 (+ 2973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:33041: BlockingRecv error: recv got EOF from 127.10.236.193:33041 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
Sep 26 13:50:45 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:45 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:45.431828 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:45.425746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:50561)
0926 13:50:45.426067 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:45.426081 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:45.426248 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:45.426507 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:45.426510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:45.426763 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:45.426906 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:45.426913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.427806 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.427811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:45.428386 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:45.428393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.428853 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.428857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:45.428941 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:45.429452 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:45.429463 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:45.431714 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":85,"threads_started":1}
W20260926 13:50:45.432078 20377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:45.425878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:50561 (local address 127.10.236.254:45309)
0926 13:50:45.426031 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:45.426035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:45.426174 (+ 139us) server_negotiation.cc:407] Connection header received
0926 13:50:45.426340 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:45.426343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:45.426399 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:45.426490 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:45.427025 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.427699 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.428505 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.428739 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.428851 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:45.431855 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:50561: BlockingRecv error: recv got EOF from 127.10.236.194:50561 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 13:50:45 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:45 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:45.636477 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:45.630349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:59273)
0926 13:50:45.630701 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:45.630720 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:45.630881 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:45.631181 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:45.631184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:45.631427 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:45.631556 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:45.631564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.632396 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.632400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:45.633209 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:45.633224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.633768 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.633773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:45.633871 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:45.634398 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:45.634410 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:45.636365 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":85,"threads_started":1}
W20260926 13:50:45.636704 20377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:45.630485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:59273 (local address 127.10.236.254:45309)
0926 13:50:45.630683 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:45.630688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:45.630814 (+ 126us) server_negotiation.cc:407] Connection header received
0926 13:50:45.630990 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:45.630994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:45.631058 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:45.631159 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:45.631689 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.632267 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.633337 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:45.633621 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:45.633741 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:45.636500 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:59273: BlockingRecv error: recv got EOF from 127.10.236.195:59273 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 26 13:50:46 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:46 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:46.241597 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:46.235659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:57729)
0926 13:50:46.236029 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:46.236043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:46.236138 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:46.236439 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:46.236441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:46.236621 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:46.236713 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:46.236718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.237481 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.237483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:46.238040 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:46.238049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.238476 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.238479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:46.238536 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:46.239067 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:46.239078 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:46.241425 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":75,"threads_started":1}
W20260926 13:50:46.241672 20381 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:46.235746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57729 (local address 127.10.236.254:45309)
0926 13:50:46.236088 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:46.236092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:46.236108 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:46.236255 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:46.236266 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:46.236314 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:46.236402 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:46.236841 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.237378 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.238157 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.238377 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.238458 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:46.241548 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:57729: BlockingRecv error: recv got EOF from 127.10.236.193:57729 (error 108)
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":139,"threads_started":1}
Sep 26 13:50:46 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:46 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:46.438357 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:46.432565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:57917)
0926 13:50:46.432862 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:46.432875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:46.432970 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:46.433255 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:46.433258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:46.433474 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:46.433605 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:46.433613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.434438 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.434441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:46.435070 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:46.435079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.435595 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.435598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:46.435668 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:46.436209 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:46.436221 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:46.438187 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":95,"threads_started":1}
W20260926 13:50:46.438452 20381 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:46.432665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:57917 (local address 127.10.236.254:45309)
0926 13:50:46.432813 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:46.432817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:46.432946 (+ 129us) server_negotiation.cc:407] Connection header received
0926 13:50:46.433056 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:46.433058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:46.433101 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:46.433179 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:46.433762 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.434312 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.435205 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.435427 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.435585 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:46.438333 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:57917: BlockingRecv error: recv got EOF from 127.10.236.194:57917 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 13:50:46 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:46 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:46.649170 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:46.637274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:45219)
0926 13:50:46.637673 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:46.637689 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:46.637800 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:46.638053 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:46.638056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:46.638251 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:46.638345 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:46.638350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.639128 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.639130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:46.639827 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:46.639836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.640305 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.640307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:46.640372 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:46.646882 (+ 6510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:46.646895 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:46.649055 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":107,"threads_started":1}
W20260926 13:50:46.649394 20381 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:46.637362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45219 (local address 127.10.236.254:45309)
0926 13:50:46.637516 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:46.637521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:46.637799 (+ 278us) server_negotiation.cc:407] Connection header received
0926 13:50:46.637883 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:46.637885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:46.637933 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:46.638014 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:46.638461 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.639027 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.639957 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:46.640194 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:46.640282 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:46.649236 (+ 8954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:45219: BlockingRecv error: recv got EOF from 127.10.236.195:45219 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 13:50:47 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:47 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:47.248654 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:47.242464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:35865)
0926 13:50:47.242765 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:47.242780 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:47.242878 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:47.243206 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:47.243209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:47.243527 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:47.243676 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:47.243684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.244802 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.244806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:47.245535 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:47.245545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.246030 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.246032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:47.246091 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:47.246655 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:47.246667 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:47.248539 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":74,"threads_started":1}
W20260926 13:50:47.248883 20385 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:47.242759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:35865 (local address 127.10.236.254:45309)
0926 13:50:47.242990 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:47.242994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:47.243006 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:50:47.243044 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:47.243046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:47.243085 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:47.243162 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:47.243832 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.244659 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.245673 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.245921 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.246035 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:47.248685 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:35865: BlockingRecv error: recv got EOF from 127.10.236.193:35865 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":81,"threads_started":1}
Sep 26 13:50:47 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:47 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:47.444613 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:47.439296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:54523)
0926 13:50:47.439566 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:47.439579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:47.439706 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:47.439958 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:47.439960 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:47.440134 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:47.440230 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:47.440235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.441184 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.441187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:47.441741 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:47.441749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.442143 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.442145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:47.442210 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:47.442701 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:47.442713 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:47.444499 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
W20260926 13:50:47.444753 20385 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:47.439339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:54523 (local address 127.10.236.254:45309)
0926 13:50:47.439532 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:47.439539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:47.439650 (+ 111us) server_negotiation.cc:407] Connection header received
0926 13:50:47.439805 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:47.439807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:47.439852 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:47.439921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:47.440329 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.441079 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.441840 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.442047 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.442120 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:47.444629 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:54523: BlockingRecv error: recv got EOF from 127.10.236.194:54523 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 13:50:47 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:47 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:47.655278 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:47.649969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:57413)
0926 13:50:47.650218 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:47.650239 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:47.650338 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:47.650567 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:47.650569 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:47.650784 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:47.650883 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:47.650888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.651724 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.651727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:47.652316 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:47.652324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.652780 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.652783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:47.652848 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:47.653332 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:47.653343 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:47.655163 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
W20260926 13:50:47.655431 20385 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:47.650016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:57413 (local address 127.10.236.254:45309)
0926 13:50:47.650158 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:47.650161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:47.650309 (+ 148us) server_negotiation.cc:407] Connection header received
0926 13:50:47.650416 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:47.650418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:47.650464 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:47.650536 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:47.650999 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.651540 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.652430 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:47.652642 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:47.652761 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:47.655307 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:57413: BlockingRecv error: recv got EOF from 127.10.236.195:57413 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 13:50:48 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:48 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:48.255061 20388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:48.249485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:55387)
0926 13:50:48.249754 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:48.249772 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:48.249875 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:48.250224 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:48.250226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:48.250405 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:48.250497 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:48.250502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.251340 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.251342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:48.251936 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:48.251945 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.252398 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.252400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:48.252459 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:48.252968 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:48.252980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:48.254946 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
W20260926 13:50:48.255321 20389 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:48.249562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:55387 (local address 127.10.236.254:45309)
0926 13:50:48.249831 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:48.249834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:48.249848 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:48.250027 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:48.250030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:48.250080 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:48.250169 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:48.250634 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.251237 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.252044 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.252298 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.252384 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:48.255161 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:55387: BlockingRecv error: recv got EOF from 127.10.236.193:55387 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":115,"threads_started":1}
Sep 26 13:50:48 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:48 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:48.451871 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:48.445633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:44503)
0926 13:50:48.445975 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:48.445996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:48.446116 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:48.446416 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:48.446418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:48.446601 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:48.446709 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:48.446715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.447611 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.447614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:48.448190 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:48.448198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.448850 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.448855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:48.448950 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:48.449573 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:48.449587 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:48.451750 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
W20260926 13:50:48.452126 20389 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:48.445740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:44503 (local address 127.10.236.254:45309)
0926 13:50:48.445900 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:48.445904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:48.446140 (+ 236us) server_negotiation.cc:407] Connection header received
0926 13:50:48.446196 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:48.446199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:48.446253 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:48.446343 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:48.446881 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.447472 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.448300 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.448641 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.448757 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:48.451980 (+ 3223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:44503: BlockingRecv error: recv got EOF from 127.10.236.194:44503 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 13:50:48 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:48 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:48.662318 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:48.656216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:38251)
0926 13:50:48.656648 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:48.656666 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:48.656829 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:48.657116 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:48.657120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:48.657372 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:48.657468 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:48.657474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.658295 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.658299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:48.658908 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:48.658915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.659394 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.659398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:48.659547 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:48.660072 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:48.660085 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:48.662169 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
W20260926 13:50:48.662400 20389 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:48.656427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:38251 (local address 127.10.236.254:45309)
0926 13:50:48.656605 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:48.656610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:48.656761 (+ 151us) server_negotiation.cc:407] Connection header received
0926 13:50:48.656956 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:48.656959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:48.657006 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:48.657095 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:48.657600 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.658173 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.659030 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:48.659276 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:48.659394 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:48.662309 (+ 2915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:38251: BlockingRecv error: recv got EOF from 127.10.236.195:38251 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 13:50:49 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:49 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:49.261641 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:49.256039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:48073)
0926 13:50:49.256328 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:49.256346 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:49.256456 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:49.256741 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:49.256744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:49.256925 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:49.257014 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:49.257020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.257782 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.257786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:49.258345 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:49.258353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.258803 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.258806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:49.258870 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:49.259369 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:49.259381 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:49.261465 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
W20260926 13:50:49.261734 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:49.256075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:48073 (local address 127.10.236.254:45309)
0926 13:50:49.256337 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:49.256343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:49.256406 (+ 63us) server_negotiation.cc:407] Connection header received
0926 13:50:49.256561 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:49.256563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:49.256606 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:49.256683 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:49.257122 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.257660 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.258453 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.258680 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.258759 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:49.261595 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:48073: BlockingRecv error: recv got EOF from 127.10.236.193:48073 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":59,"threads_started":1}
Sep 26 13:50:49 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:49 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:49.458357 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:49.452766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:54119)
0926 13:50:49.453050 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:49.453063 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:49.453163 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:49.453372 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:49.453374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:49.453549 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:49.453690 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:49.453695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.454417 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.454420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:49.455024 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:49.455033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.455600 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.455604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:49.455688 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:49.456269 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:49.456283 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:49.458246 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":116,"threads_started":1}
W20260926 13:50:49.458564 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:49.452817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:54119 (local address 127.10.236.254:45309)
0926 13:50:49.452994 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:49.452998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:49.453144 (+ 146us) server_negotiation.cc:407] Connection header received
0926 13:50:49.453247 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:49.453249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:49.453291 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:49.453365 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:49.453805 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.454311 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.455148 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.455416 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.455558 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:49.458431 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:54119: BlockingRecv error: recv got EOF from 127.10.236.194:54119 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 13:50:49 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:49 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:49.668953 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:49.663291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:45301)
0926 13:50:49.663659 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:49.663684 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:49.663787 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:49.664142 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:49.664145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:49.664355 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:49.664453 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:49.664459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.665249 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.665252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:49.665871 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:49.665880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.666308 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.666311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:49.666374 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:49.666910 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:49.666921 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:49.668808 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":65,"threads_started":1}
W20260926 13:50:49.669064 20393 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:49.663548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45301 (local address 127.10.236.254:45309)
0926 13:50:49.663698 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:49.663701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:49.663741 (+ 40us) server_negotiation.cc:407] Connection header received
0926 13:50:49.663928 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:49.663930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:49.664002 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:49.664089 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:49.664614 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.665139 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.665981 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:49.666183 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:49.666271 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:49.668924 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:45301: BlockingRecv error: recv got EOF from 127.10.236.195:45301 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 13:50:50 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:50 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:50.268565 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:50.262611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:46405)
0926 13:50:50.262897 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:50.262918 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:50.263010 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:50.263265 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:50.263268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:50.263471 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:50.263619 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:50.263626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.264486 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.264489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:50.265051 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:50.265059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.265533 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.265536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:50.265625 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:50.266222 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:50.266250 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:50.268374 (+ 2124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
W20260926 13:50:50.268646 20397 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:50.262751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:46405 (local address 127.10.236.254:45309)
0926 13:50:50.263005 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:50.263008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:50.263022 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:50.263080 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:50.263082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:50.263122 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:50.263199 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:50.263776 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.264317 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.265174 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.265412 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.265528 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:50.268514 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:46405: BlockingRecv error: recv got EOF from 127.10.236.193:46405 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
Sep 26 13:50:50 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:50 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:50.464737 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:50.459117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:35719)
0926 13:50:50.459433 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:50.459450 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:50.459609 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:50.459877 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:50.459880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:50.460091 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:50.460180 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:50.460186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.460974 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.460979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:50.461552 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:50.461559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.462015 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.462019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:50.462106 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:50.462640 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:50.462652 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:50.464607 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":78,"threads_started":1}
W20260926 13:50:50.464861 20397 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:50.459239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:35719 (local address 127.10.236.254:45309)
0926 13:50:50.459401 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:50.459406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:50.459549 (+ 143us) server_negotiation.cc:407] Connection header received
0926 13:50:50.459708 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:50.459712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:50.459768 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:50.459853 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:50.460305 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.460862 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.461672 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.461903 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.462010 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:50.464707 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:35719: BlockingRecv error: recv got EOF from 127.10.236.194:35719 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 13:50:50 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:50 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:50.675387 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:50.669907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:45351)
0926 13:50:50.670215 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:50.670228 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:50.670333 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:50.670583 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:50.670586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:50.670819 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:50.670910 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:50.670916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.671790 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.671793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:50.672347 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:50.672354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.672806 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.672810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:50.672894 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:50.673456 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:50.673468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:50.675202 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":81,"threads_started":1}
W20260926 13:50:50.675423 20397 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:50.669997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:45351 (local address 127.10.236.254:45309)
0926 13:50:50.670189 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:50.670194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:50.670284 (+ 90us) server_negotiation.cc:407] Connection header received
0926 13:50:50.670417 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:50.670420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:50.670463 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:50.670529 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:50.671038 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.671646 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.672476 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:50.672696 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:50.672806 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:50.675323 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:45351: BlockingRecv error: recv got EOF from 127.10.236.195:45351 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Sep 26 13:50:51 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:51 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:51.274665 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:51.269386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:49533)
0926 13:50:51.269666 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:51.269687 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:51.269784 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:51.270020 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:51.270022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:51.270201 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:51.270296 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:51.270301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.271005 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.271007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:51.271573 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:51.271580 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.271978 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.271980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:51.272038 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:51.272494 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:51.272506 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:51.274505 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
W20260926 13:50:51.274837 20401 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:51.269496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:49533 (local address 127.10.236.254:45309)
0926 13:50:51.269730 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:51.269734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:51.269748 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:51.269854 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:51.269856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:51.269898 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:51.269980 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:51.270398 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.270914 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.271667 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.271881 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.271959 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:51.274669 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:49533: BlockingRecv error: recv got EOF from 127.10.236.193:49533 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
Sep 26 13:50:51 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:51 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:51.471127 20402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:51.465446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:56237)
0926 13:50:51.465709 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:51.465723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:51.465848 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:51.466086 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:51.466088 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:51.466267 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:51.466363 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:51.466369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.467307 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.467311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:51.467917 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:51.467927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.468350 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.468353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:51.468419 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:51.469001 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:51.469013 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:51.471011 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
W20260926 13:50:51.471321 20401 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:51.465569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:56237 (local address 127.10.236.254:45309)
0926 13:50:51.465716 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:51.465720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:51.465792 (+ 72us) server_negotiation.cc:407] Connection header received
0926 13:50:51.465932 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:51.465934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:51.465978 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:51.466050 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:51.466520 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.467183 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.468038 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.468247 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.468331 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:51.471178 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:56237: BlockingRecv error: recv got EOF from 127.10.236.194:56237 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 13:50:51 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:51 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:51.687417 20403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:51.676155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:42743)
0926 13:50:51.676456 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:51.676471 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:51.676576 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:51.676818 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:51.676821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:51.677012 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:51.677106 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:51.677112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.677883 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.677886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:51.678509 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:51.678517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.678955 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.678958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:51.679025 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:51.684896 (+ 5871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:51.684911 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:51.687289 (+ 2378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":118,"threads_started":1}
W20260926 13:50:51.687657 20401 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:51.676252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:42743 (local address 127.10.236.254:45309)
0926 13:50:51.676394 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:51.676401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:51.676544 (+ 143us) server_negotiation.cc:407] Connection header received
0926 13:50:51.676655 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:51.676657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:51.676700 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:51.676774 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:51.677231 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.677740 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.678608 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:51.678848 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:51.678920 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:51.687529 (+ 8609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:42743: BlockingRecv error: recv got EOF from 127.10.236.195:42743 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 13:50:52 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:52 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:52.281845 20404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:52.275684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:52905)
0926 13:50:52.275977 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:52.275998 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:52.276100 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:52.276386 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:52.276389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:52.276594 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:52.276733 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:52.276740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.277563 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.277566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:52.278166 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:52.278174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.278815 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.278817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:52.279010 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:52.279746 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:52.279767 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:52.281730 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
W20260926 13:50:52.282079 20405 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:52.275769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:52905 (local address 127.10.236.254:45309)
0926 13:50:52.276088 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:52.276093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:52.276109 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:50:52.276188 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:52.276191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:52.276235 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:52.276308 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:52.276912 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.277444 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.278306 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.278674 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.278804 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:52.281938 (+ 3134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:52905: BlockingRecv error: recv got EOF from 127.10.236.193:52905 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":78,"threads_started":1}
Sep 26 13:50:52 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:52 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:52.477496 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:52.471925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:34387)
0926 13:50:52.472279 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:52.472297 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:52.472456 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:52.472774 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:52.472776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:52.472993 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:52.473079 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:52.473085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.473839 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.473845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:52.474412 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:52.474419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.474871 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.474875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:52.474960 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:52.475546 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:52.475558 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:52.477382 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":96,"threads_started":1}
W20260926 13:50:52.477641 20405 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:52.472107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:34387 (local address 127.10.236.254:45309)
0926 13:50:52.472279 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:52.472283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:52.472391 (+ 108us) server_negotiation.cc:407] Connection header received
0926 13:50:52.472559 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:52.472563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:52.472616 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:52.472759 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:52.473202 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.473729 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.474532 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.474760 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.474866 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:52.477522 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:34387: BlockingRecv error: recv got EOF from 127.10.236.194:34387 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 26 13:50:52 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:52 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:52.693856 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:52.688215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:58749)
0926 13:50:52.688586 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:52.688600 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:52.688777 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:52.689060 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:52.689063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:52.689275 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:52.689363 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:52.689369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.690116 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.690122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:52.690680 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:52.690687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.691158 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.691161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:52.691246 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:52.691790 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:52.691802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:52.693683 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":84,"threads_started":1}
W20260926 13:50:52.693898 20405 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:52.688347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:58749 (local address 127.10.236.254:45309)
0926 13:50:52.688536 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:52.688541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:52.688682 (+ 141us) server_negotiation.cc:407] Connection header received
0926 13:50:52.688884 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:52.688888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:52.688943 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:52.689030 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:52.689487 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.690007 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.690807 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:52.691033 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:52.691142 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:52.693799 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:58749: BlockingRecv error: recv got EOF from 127.10.236.195:58749 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 13:50:53 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:53 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:53.288909 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:53.282743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:59401)
0926 13:50:53.283097 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:53.283119 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:53.283211 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:53.283436 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:53.283437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:53.283707 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:53.283801 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:53.283807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.284671 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.284675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:53.285260 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:53.285270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.285902 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.285906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:53.285980 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:53.286506 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:53.286525 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:53.288756 (+ 2231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":125,"threads_started":1}
W20260926 13:50:53.289119 20409 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:53.282893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:59401 (local address 127.10.236.254:45309)
0926 13:50:53.283218 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:53.283222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:53.283263 (+ 41us) server_negotiation.cc:407] Connection header received
0926 13:50:53.283304 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:53.283306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:53.283350 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:53.283425 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:53.283937 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.284503 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.285441 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.285733 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.285819 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:53.288985 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:59401: BlockingRecv error: recv got EOF from 127.10.236.193:59401 (error 108)
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":74,"threads_started":1}
Sep 26 13:50:53 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:53 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:53.484406 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:53.478487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:51239)
0926 13:50:53.478920 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:53.478934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:53.479038 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:53.479277 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:53.479279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:53.479463 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:53.479589 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:53.479595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.480475 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.480481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:53.481103 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:53.481113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.481572 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.481576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:53.481659 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:53.482234 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:53.482246 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:53.484277 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":218,"threads_started":1}
W20260926 13:50:53.484583 20409 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:53.478599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:51239 (local address 127.10.236.254:45309)
0926 13:50:53.478755 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:53.478759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:53.479032 (+ 273us) server_negotiation.cc:407] Connection header received
0926 13:50:53.479109 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:53.479112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:53.479159 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:53.479236 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:53.479725 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.480359 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.481223 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.481456 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.481569 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:53.484424 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:51239: BlockingRecv error: recv got EOF from 127.10.236.194:51239 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 13:50:53 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:53 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.195@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:53.700338 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:53.694613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.195:43251)
0926 13:50:53.694901 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:53.694917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:53.695027 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:53.695293 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:53.695295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:53.695539 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:53.695652 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:53.695659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.696450 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.696453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:53.697076 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:53.697093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.697554 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.697557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:53.697642 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:53.698156 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:53.698168 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:53.700127 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":88,"threads_started":1}
W20260926 13:50:53.700481 20409 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:53.694767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:43251 (local address 127.10.236.254:45309)
0926 13:50:53.694929 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:53.694932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:53.695018 (+ 86us) server_negotiation.cc:407] Connection header received
0926 13:50:53.695127 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:53.695130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:53.695175 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:53.695240 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:53.695788 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.696336 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.697218 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:53.697439 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:53.697526 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:53.700360 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.195:43251: BlockingRecv error: recv got EOF from 127.10.236.195:43251 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 13:50:54 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:54 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.193@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:54.295680 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:54.289876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.193:47751)
0926 13:50:54.290159 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:54.290182 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:54.290269 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:54.290569 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:54.290572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:54.290760 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:54.290885 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:54.290891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.291751 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.291754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:54.292357 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:54.292368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.292951 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.292954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:54.293022 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:54.293594 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:54.293607 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:54.295551 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":115,"threads_started":1}
W20260926 13:50:54.295862 20413 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:54.290016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:47751 (local address 127.10.236.254:45309)
0926 13:50:54.290332 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:54.290336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:54.290350 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:50:54.290403 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:54.290405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:54.290445 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:54.290532 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:54.291028 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.291616 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.292535 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.292806 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.292917 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:54.295702 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.193:47751: BlockingRecv error: recv got EOF from 127.10.236.193:47751 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
Sep 26 13:50:54 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:50:54 dist-test-slave-r34j krb5kdc[19355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.236.194@KRBTEST.COM for oryx/127.10.236.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:50:54.490948 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:54.485210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:45309 (local address 127.10.236.194:50755)
0926 13:50:54.485526 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:54.485540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:54.485664 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:54.485961 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:54.485964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:54.486183 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:54.486271 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:54.486277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.487108 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.487114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:54.487763 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:54.487772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.488270 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.488273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:54.488368 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:54.488930 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:54.488942 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:54.490778 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.236.254:45309: Server oryx/127.10.236.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":86,"threads_started":1}
W20260926 13:50:54.491012 20413 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:54.485315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:50755 (local address 127.10.236.254:45309)
0926 13:50:54.485476 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:54.485480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:54.485606 (+ 126us) server_negotiation.cc:407] Connection header received
0926 13:50:54.485768 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:54.485776 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:54.485821 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:54.485939 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:54.486401 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.486978 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.487902 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.488149 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.488265 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:54.490906 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.236.194:50755: BlockingRecv error: recv got EOF from 127.10.236.194:50755 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:50:54.587146 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19938
I20260926 13:50:54.593837 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20090
I20260926 13:50:54.599220 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20222
I20260926 13:50:54.604578 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 19868
2026-09-26T13:50:54Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18113 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T13:50:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:54Z Disabled control of system clock
I20260926 13:50:54.635751 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:36291
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:36291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 13:50:54.744939 20418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:54.745203 20418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:54.745236 20418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:54.749118 20418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:54.749166 20418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:54.749181 20418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:54.749197 20418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:54.749210 20418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:54.753931 20418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:36291
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:36291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:54.754921 20418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:54.756232 20418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:54.764952 20426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:54.764948 20424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:54.765031 20423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:54.765025 20418 server_base.cc:1034] running on GCE node
I20260926 13:50:54.765793 20418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:54.766935 20418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:54.768129 20418 hybrid_clock.cc:648] HybridClock initialized: now 1790430654768118 us; error 32 us; skew 500 ppm
I20260926 13:50:54.770046 20418 webserver.cc:460] Webserver started at http://127.10.236.254:44295/ using document root <none> and password file <none>
I20260926 13:50:54.770675 20418 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:54.770766 20418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:54.771000 20418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:54.773138 20418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "d8d895e0d77d4e7599a400937781366f"
format_stamp: "Formatted at 2026-09-26 13:50:54 on dist-test-slave-r34j"
server_key: "b656acc6748f3f45da2a1d11573281dc"
server_key_iv: "522f98fa681f7a6a5ce447ca9c5cddbe"
server_key_version: "encryptionkey@0"
I20260926 13:50:54.773728 20418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "d8d895e0d77d4e7599a400937781366f"
format_stamp: "Formatted at 2026-09-26 13:50:54 on dist-test-slave-r34j"
server_key: "b656acc6748f3f45da2a1d11573281dc"
server_key_iv: "522f98fa681f7a6a5ce447ca9c5cddbe"
server_key_version: "encryptionkey@0"
I20260926 13:50:54.777910 20418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:54.780522 20432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:54.781826 20418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:54.781986 20418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "d8d895e0d77d4e7599a400937781366f"
format_stamp: "Formatted at 2026-09-26 13:50:54 on dist-test-slave-r34j"
server_key: "b656acc6748f3f45da2a1d11573281dc"
server_key_iv: "522f98fa681f7a6a5ce447ca9c5cddbe"
server_key_version: "encryptionkey@0"
I20260926 13:50:54.782114 20418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:54.792761 20418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:54.793466 20418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:54.793666 20418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:54.800663 20418 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:36291
I20260926 13:50:54.800702 20484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:36291 every 8 connection(s)
I20260926 13:50:54.801668 20418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:54.805083 20485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:54.810714 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f: Bootstrap starting.
I20260926 13:50:54.811597 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20418
I20260926 13:50:54.811728 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:54.812012 11187 external_mini_cluster.cc:1428] Setting key 9c7c86ec5ea5156ff000373b7d18abf6
I20260926 13:50:54.813655 20485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:54.814393 20485 log.cc:824] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:54.816275 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f: No bootstrap required, opened a new log
I20260926 13:50:54.819684 20485 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d895e0d77d4e7599a400937781366f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36291 } }
I20260926 13:50:54.820048 20485 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:54.820163 20485 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8d895e0d77d4e7599a400937781366f, State: Initialized, Role: FOLLOWER
I20260926 13:50:54.820678 20485 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d895e0d77d4e7599a400937781366f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36291 } }
I20260926 13:50:54.820852 20485 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:54.820937 20485 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:54.821028 20485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:54.821506 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:54.813131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.10.236.254:36291)
0926 13:50:54.813842 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:54.813857 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:54.813892 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:50:54.814486 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:54.814502 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:54.814946 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:54.815281 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:54.815732 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.816766 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.817656 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:54.818134 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:54.818309 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:54.819199 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:54.819204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:54.819216 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:54.819669 (+ 453us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:54.819725 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:54.819745 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:54.819872 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:54.820213 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 13:50:54.820509 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":147,"threads_started":1}
I20260926 13:50:54.822139 20485 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d895e0d77d4e7599a400937781366f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36291 } }
I20260926 13:50:54.822513 20485 leader_election.cc:302] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8d895e0d77d4e7599a400937781366f; no voters:
I20260926 13:50:54.822865 20485 leader_election.cc:288] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:54.823004 20490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:54.823254 20490 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [term 1 LEADER]: Becoming Leader. State: Replica: d8d895e0d77d4e7599a400937781366f, State: Running, Role: LEADER
I20260926 13:50:54.823686 20490 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d895e0d77d4e7599a400937781366f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36291 } }
I20260926 13:50:54.824151 20485 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:54.825465 20491 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8d895e0d77d4e7599a400937781366f. Latest consensus state: current_term: 1 leader_uuid: "d8d895e0d77d4e7599a400937781366f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d895e0d77d4e7599a400937781366f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36291 } } }
I20260926 13:50:54.825500 20492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8d895e0d77d4e7599a400937781366f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d895e0d77d4e7599a400937781366f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36291 } } }
I20260926 13:50:54.825690 20491 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:54.825723 20492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f [sys.catalog]: This master's current role is: LEADER
W20260926 13:50:54.829046 20505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:50:54.829131 20505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:50:54.829268 20501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:54.830068 20501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:54.835781 20501 catalog_manager.cc:1347] Generated new cluster ID: 7bdfce06c3a14cd6a447a42cef59affb
I20260926 13:50:54.835861 20501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:54.851141 20501 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:54.852139 20501 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:54.870798 20501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8d895e0d77d4e7599a400937781366f: Generated new TSK 0
I20260926 13:50:54.871610 20501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:54.881393 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36291
--builtin_ntp_servers=127.10.236.212:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 13:50:54.995874 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:54.996261 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:54.996368 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:55.000559 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:55.000639 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:55.000725 20509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:55.005741 20509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:55.006896 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:55.008414 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:55.018723 20514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:55.018771 20517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:55.018762 20515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.018823 20509 server_base.cc:1034] running on GCE node
I20260926 13:50:55.019420 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:55.020159 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:55.021335 20509 hybrid_clock.cc:648] HybridClock initialized: now 1790430655021317 us; error 30 us; skew 500 ppm
I20260926 13:50:55.023406 20509 webserver.cc:460] Webserver started at http://127.10.236.193:41559/ using document root <none> and password file <none>
I20260926 13:50:55.024111 20509 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:55.024175 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:55.024412 20509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:55.026728 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "499a9015992b49118d110098656238f5"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "c487321f739ee3274653a5cc50b28040"
server_key_iv: "1cd4e0fc854aa4e9cc096a564a79865f"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.027295 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "499a9015992b49118d110098656238f5"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "c487321f739ee3274653a5cc50b28040"
server_key_iv: "1cd4e0fc854aa4e9cc096a564a79865f"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.030977 20509 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:55.033601 20523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.034982 20509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:55.035127 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "499a9015992b49118d110098656238f5"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "c487321f739ee3274653a5cc50b28040"
server_key_iv: "1cd4e0fc854aa4e9cc096a564a79865f"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.035248 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:55.048382 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:55.049047 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:55.049243 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:55.049880 20509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:55.050803 20509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:55.050875 20509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.050966 20509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:55.051016 20509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.059739 20509 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:40087
I20260926 13:50:55.059753 20636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:40087 every 8 connection(s)
I20260926 13:50:55.060643 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:55.068698 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20509
I20260926 13:50:55.068832 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:55.069196 11187 external_mini_cluster.cc:1428] Setting key eead183559b4c90d6c798fe67a98aa6a
I20260926 13:50:55.070015 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.062509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:54305 (local address 127.10.236.254:36291)
0926 13:50:55.062651 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:55.062654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:55.063721 (+ 1067us) server_negotiation.cc:407] Connection header received
0926 13:50:55.065126 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:55.065129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:55.065177 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:55.065257 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:55.066151 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.066672 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.067876 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.068168 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.068251 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:55.069515 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:55.069518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:55.069520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:55.069532 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:55.069552 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:55.069559 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:55.069645 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:55.069814 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 13:50:55.069878 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:50:55.070992 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.062852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36291 (local address 127.10.236.193:54305)
0926 13:50:55.063558 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:55.063606 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:55.064883 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:55.065440 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:55.065447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:55.065532 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:55.066020 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:55.066032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.066777 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.066780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:55.067745 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:55.067754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.068302 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.068305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:55.068546 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:55.069201 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:55.069205 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:55.069262 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:55.069356 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:55.069650 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:55.069661 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:55.069669 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:50:55.069836 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 13:50:55.070121 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":107,"threads_started":1}
I20260926 13:50:55.071291 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36291
--builtin_ntp_servers=127.10.236.212:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 13:50:55.072309 20637 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36291
I20260926 13:50:55.072578 20637 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:55.073103 20637 heartbeater.cc:511] Master 127.10.236.254:36291 requested a full tablet report, sending...
I20260926 13:50:55.074580 20449 ts_manager.cc:194] Registered new tserver with Master: 499a9015992b49118d110098656238f5 (127.10.236.193:40087)
I20260926 13:50:55.075820 20449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.193:54305
W20260926 13:50:55.181092 20640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:55.181383 20640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:55.181453 20640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:55.185514 20640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:55.185593 20640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:55.185717 20640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:55.190582 20640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:55.191746 20640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:55.193109 20640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:55.202734 20645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:55.202718 20648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.202890 20640 server_base.cc:1034] running on GCE node
W20260926 13:50:55.202718 20646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.203419 20640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:55.204157 20640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:55.205363 20640 hybrid_clock.cc:648] HybridClock initialized: now 1790430655205346 us; error 26 us; skew 500 ppm
I20260926 13:50:55.207217 20640 webserver.cc:460] Webserver started at http://127.10.236.194:41823/ using document root <none> and password file <none>
I20260926 13:50:55.207865 20640 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:55.207954 20640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:55.208165 20640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:55.210211 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "306e8e2ab2af4aef814a7a0c2b993c95"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "dc9e4ceeaf679d6b8bdb674c8a62c827"
server_key_iv: "9cccfc22a8766428ec7f7d91a753ea8d"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.210767 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "306e8e2ab2af4aef814a7a0c2b993c95"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "dc9e4ceeaf679d6b8bdb674c8a62c827"
server_key_iv: "9cccfc22a8766428ec7f7d91a753ea8d"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.214612 20640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260926 13:50:55.217175 20654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.218405 20640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:50:55.218535 20640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "306e8e2ab2af4aef814a7a0c2b993c95"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "dc9e4ceeaf679d6b8bdb674c8a62c827"
server_key_iv: "9cccfc22a8766428ec7f7d91a753ea8d"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.218652 20640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:55.240259 20640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:55.240948 20640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:55.241096 20640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:55.241619 20640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:55.242506 20640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:55.242549 20640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.242588 20640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:55.242604 20640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.250991 20640 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:41999
I20260926 13:50:55.251060 20767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:41999 every 8 connection(s)
I20260926 13:50:55.251994 20640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:55.258159 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20640
I20260926 13:50:55.258309 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:55.258618 11187 external_mini_cluster.cc:1428] Setting key f6b466c4854db741a1f14d66a048e20d
I20260926 13:50:55.260715 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36291
--builtin_ntp_servers=127.10.236.212:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 13:50:55.261333 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.253834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:56277 (local address 127.10.236.254:36291)
0926 13:50:55.253974 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:55.253978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:55.254980 (+ 1002us) server_negotiation.cc:407] Connection header received
0926 13:50:55.256431 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:55.256434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:55.256491 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:55.256601 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:55.257479 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.258117 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.259410 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.259686 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.259800 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:55.260848 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:55.260854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:55.260860 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:55.260876 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:55.260895 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:55.260905 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:55.260964 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:55.261135 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 13:50:55.261190 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:50:55.262168 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.254156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36291 (local address 127.10.236.194:56277)
0926 13:50:55.254813 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:55.254855 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:55.256213 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:55.256772 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:55.256779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:55.256863 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:55.257324 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:55.257335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.258292 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.258296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:55.259277 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:55.259286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.259840 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.259844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:55.260003 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:55.260603 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:55.260604 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:55.260641 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:55.260700 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:55.260983 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:55.260991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:55.260998 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:50:55.261125 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 13:50:55.261389 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260926 13:50:55.263533 20768 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36291
I20260926 13:50:55.263773 20768 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:55.264389 20768 heartbeater.cc:511] Master 127.10.236.254:36291 requested a full tablet report, sending...
I20260926 13:50:55.265528 20449 ts_manager.cc:194] Registered new tserver with Master: 306e8e2ab2af4aef814a7a0c2b993c95 (127.10.236.194:41999)
I20260926 13:50:55.266106 20449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.194:56277
W20260926 13:50:55.374373 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:55.374632 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:55.374672 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:55.378707 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:55.378769 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:55.378826 20771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:55.383533 20771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:55.384651 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:55.386118 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:55.395727 20777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.395879 20771 server_base.cc:1034] running on GCE node
W20260926 13:50:55.395762 20776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:55.396114 20779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.396535 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:55.397193 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:55.398365 20771 hybrid_clock.cc:648] HybridClock initialized: now 1790430655398327 us; error 43 us; skew 500 ppm
I20260926 13:50:55.400261 20771 webserver.cc:460] Webserver started at http://127.10.236.195:37685/ using document root <none> and password file <none>
I20260926 13:50:55.400835 20771 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:55.400903 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:55.401155 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:55.403183 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "6b39443d94a947bc923fb74fe0120e84"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "779ffabe5706395bd2c83f2d8602cda0"
server_key_iv: "0032c2fc4550951798013de66e2c574e"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.403757 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "6b39443d94a947bc923fb74fe0120e84"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "779ffabe5706395bd2c83f2d8602cda0"
server_key_iv: "0032c2fc4550951798013de66e2c574e"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.407564 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:50:55.409999 20785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.411194 20771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:55.411314 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "6b39443d94a947bc923fb74fe0120e84"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "779ffabe5706395bd2c83f2d8602cda0"
server_key_iv: "0032c2fc4550951798013de66e2c574e"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.411422 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:55.442132 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:55.442860 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:55.443064 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:55.443691 20771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:55.444666 20771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:55.444712 20771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.444752 20771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:55.444767 20771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.453565 20771 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:33839
I20260926 13:50:55.453590 20898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:33839 every 8 connection(s)
I20260926 13:50:55.454487 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:55.457899 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20771
I20260926 13:50:55.457988 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:55.458256 11187 external_mini_cluster.cc:1428] Setting key 5db5d0947d2c1371f8e21507ac28e78a
I20260926 13:50:55.463771 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.456346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:53853 (local address 127.10.236.254:36291)
0926 13:50:55.456496 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:55.456499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:55.457515 (+ 1016us) server_negotiation.cc:407] Connection header received
0926 13:50:55.458976 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:55.458979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:55.459034 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:55.459119 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:55.460147 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.460669 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.461797 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.462030 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.462140 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:55.463265 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:55.463277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:55.463280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:55.463291 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:55.463309 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:55.463317 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:55.463395 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:55.463574 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 13:50:55.463634 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 13:50:55.464651 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.456738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36291 (local address 127.10.236.195:53853)
0926 13:50:55.457372 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:55.457410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:55.458764 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:55.459296 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:55.459302 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:55.459391 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:55.459985 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:55.459999 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.460794 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.460797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:55.461672 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:55.461680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:55.462177 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:55.462181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:55.462359 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:55.463013 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:55.463015 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:55.463055 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:55.463114 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:55.463408 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:55.463417 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:55.463426 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 13:50:55.463573 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 13:50:55.463846 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
I20260926 13:50:55.465780 20899 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36291
I20260926 13:50:55.465996 20899 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:55.466450 20899 heartbeater.cc:511] Master 127.10.236.254:36291 requested a full tablet report, sending...
I20260926 13:50:55.467402 20449 ts_manager.cc:194] Registered new tserver with Master: 6b39443d94a947bc923fb74fe0120e84 (127.10.236.195:33839)
I20260926 13:50:55.467995 20449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.195:53853
I20260926 13:50:55.472031 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 13:50:55.475060 20488 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:55.474073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.10.236.254:36291)
0926 13:50:55.474252 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:55.474256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:55.474407 (+ 151us) server_negotiation.cc:407] Connection header received
0926 13:50:55.474558 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:55.474561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:55.474609 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:55.474696 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:55.474965 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50062: BlockingRecv error: recv got EOF from 127.0.0.1:50062 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 13:50:55.475399 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20509
I20260926 13:50:55.480826 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20640
I20260926 13:50:55.485925 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20771
I20260926 13:50:55.491072 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20418
2026-09-26T13:50:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (884 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T13:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:55Z Disabled control of system clock
I20260926 13:50:55.520032 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34073
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:34073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 13:50:55.641062 20910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:55.641336 20910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:55.641373 20910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:55.645433 20910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:55.645498 20910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:55.645514 20910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:55.645530 20910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:55.645545 20910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:55.650442 20910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:34073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:55.651551 20910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:55.652899 20910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:55.661521 20916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:55.661552 20915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.661628 20910 server_base.cc:1034] running on GCE node
W20260926 13:50:55.661552 20918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.662406 20910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:55.663550 20910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:55.664733 20910 hybrid_clock.cc:648] HybridClock initialized: now 1790430655664702 us; error 41 us; skew 500 ppm
I20260926 13:50:55.666536 20910 webserver.cc:460] Webserver started at http://127.10.236.254:35075/ using document root <none> and password file <none>
I20260926 13:50:55.667104 20910 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:55.667166 20910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:55.667411 20910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:55.669453 20910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "043e1837c2c74d6686454d306c2b8426"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "c9cec766da76fb8bd768b5e36f2ffb5c"
server_key_iv: "b39333d816ec4c090a8794b8f54cedc8"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.670014 20910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "043e1837c2c74d6686454d306c2b8426"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "c9cec766da76fb8bd768b5e36f2ffb5c"
server_key_iv: "b39333d816ec4c090a8794b8f54cedc8"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.673627 20910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:50:55.675972 20924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.677152 20910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:55.677275 20910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "043e1837c2c74d6686454d306c2b8426"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "c9cec766da76fb8bd768b5e36f2ffb5c"
server_key_iv: "b39333d816ec4c090a8794b8f54cedc8"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.677387 20910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:55.681977 20910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:55.682555 20910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:55.682736 20910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:55.689458 20910 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:34073
I20260926 13:50:55.689473 20976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:34073 every 8 connection(s)
I20260926 13:50:55.690363 20910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:55.693508 20977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:55.695559 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 20910
I20260926 13:50:55.695670 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:55.695988 11187 external_mini_cluster.cc:1428] Setting key e3e4ed4cf05cd1a1fd429fc94505d176
I20260926 13:50:55.699536 20977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426: Bootstrap starting.
I20260926 13:50:55.701745 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.697491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.10.236.254:34073)
0926 13:50:55.698168 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:55.698183 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:55.698217 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:50:55.698847 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:55.698864 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:55.699231 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:55.699710 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:55.700520 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:55.700525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:55.700537 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:55.700547 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:55.700604 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:55.700620 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:55.700769 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:55.700838 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 13:50:55.701075 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":124,"threads_started":1}
I20260926 13:50:55.701923 20977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:55.702696 20977 log.cc:824] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:55.704720 20977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426: No bootstrap required, opened a new log
I20260926 13:50:55.708299 20977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e1837c2c74d6686454d306c2b8426" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34073 } }
I20260926 13:50:55.708581 20977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:55.708652 20977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043e1837c2c74d6686454d306c2b8426, State: Initialized, Role: FOLLOWER
I20260926 13:50:55.709090 20977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e1837c2c74d6686454d306c2b8426" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34073 } }
I20260926 13:50:55.709240 20977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:55.709307 20977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:55.709381 20977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:55.710219 20977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e1837c2c74d6686454d306c2b8426" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34073 } }
I20260926 13:50:55.710496 20977 leader_election.cc:302] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 043e1837c2c74d6686454d306c2b8426; no voters:
I20260926 13:50:55.710732 20977 leader_election.cc:288] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:55.710880 20982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:55.711144 20982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [term 1 LEADER]: Becoming Leader. State: Replica: 043e1837c2c74d6686454d306c2b8426, State: Running, Role: LEADER
I20260926 13:50:55.711504 20982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e1837c2c74d6686454d306c2b8426" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34073 } }
I20260926 13:50:55.712001 20977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:55.713063 20984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 043e1837c2c74d6686454d306c2b8426. Latest consensus state: current_term: 1 leader_uuid: "043e1837c2c74d6686454d306c2b8426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e1837c2c74d6686454d306c2b8426" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34073 } } }
I20260926 13:50:55.713181 20984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:55.713451 20983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "043e1837c2c74d6686454d306c2b8426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e1837c2c74d6686454d306c2b8426" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34073 } } }
I20260926 13:50:55.713544 20983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:55.713675 20991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:55.716694 20991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:55.722337 20991 catalog_manager.cc:1347] Generated new cluster ID: ed4000a70d98428b9efe71f026191dc0
I20260926 13:50:55.722426 20991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:55.744685 20991 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:55.745090 20991 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:55.750563 20991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 043e1837c2c74d6686454d306c2b8426: Generated new TSK 0
I20260926 13:50:55.751264 20991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:50:55.758437 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34073
--builtin_ntp_servers=127.10.236.212:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 13:50:55.872484 21001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:55.872779 21001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:55.872864 21001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:55.877023 21001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:55.877091 21001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:55.877151 21001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:55.882351 21001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:55.883565 21001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:55.884999 21001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:55.894563 21007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:55.894497 21009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.894731 21001 server_base.cc:1034] running on GCE node
W20260926 13:50:55.894526 21006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:55.895291 21001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:55.895975 21001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:55.897230 21001 hybrid_clock.cc:648] HybridClock initialized: now 1790430655897218 us; error 32 us; skew 500 ppm
I20260926 13:50:55.899269 21001 webserver.cc:460] Webserver started at http://127.10.236.193:44387/ using document root <none> and password file <none>
I20260926 13:50:55.899943 21001 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:55.900034 21001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:55.900261 21001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:55.902354 21001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "ef10046c9b194ce88f666e249663bff5"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "57736b062dc407e89e8f434c9b79f9c8"
server_key_iv: "49e83a4f9d0d0e20e135d8e0958db763"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.902918 21001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "ef10046c9b194ce88f666e249663bff5"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "57736b062dc407e89e8f434c9b79f9c8"
server_key_iv: "49e83a4f9d0d0e20e135d8e0958db763"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.906883 21001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:50:55.909621 21015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.911127 21001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:50:55.911310 21001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "ef10046c9b194ce88f666e249663bff5"
format_stamp: "Formatted at 2026-09-26 13:50:55 on dist-test-slave-r34j"
server_key: "57736b062dc407e89e8f434c9b79f9c8"
server_key_iv: "49e83a4f9d0d0e20e135d8e0958db763"
server_key_version: "encryptionkey@0"
I20260926 13:50:55.911441 21001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:55.916481 21001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:55.917106 21001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:55.917315 21001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:55.917848 21001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:55.918771 21001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:55.918839 21001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.918912 21001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:55.918955 21001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:55.927666 21001 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:34509
I20260926 13:50:55.927726 21128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:34509 every 8 connection(s)
I20260926 13:50:55.928604 21001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:55.934687 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.930408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:38661 (local address 127.10.236.254:34073)
0926 13:50:55.930554 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:55.930558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:55.931643 (+ 1085us) server_negotiation.cc:407] Connection header received
0926 13:50:55.932995 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:55.932998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:55.933042 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:55.933156 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:55.934230 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:55.934234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:55.934236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:55.934248 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:55.934268 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:55.934285 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:55.934381 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:55.934511 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 13:50:55.934554 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:50:55.935017 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21001
I20260926 13:50:55.935151 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:55.935441 11187 external_mini_cluster.cc:1428] Setting key 7d59412c07ee2dc2b4a56966b153d3e2
I20260926 13:50:55.935653 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:55.930828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34073 (local address 127.10.236.193:38661)
0926 13:50:55.931443 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:55.931539 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:55.932796 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:55.933308 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:55.933314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:55.933399 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:55.933987 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:55.933988 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:55.934027 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:55.934089 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:55.934364 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:55.934372 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:55.934377 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:50:55.934552 (+ 175us) client_negotiation.cc:240] Negotiation successful
0926 13:50:55.934786 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":98,"threads_started":1}
I20260926 13:50:55.936856 21129 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34073
I20260926 13:50:55.937078 21129 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:55.937505 21129 heartbeater.cc:511] Master 127.10.236.254:34073 requested a full tablet report, sending...
I20260926 13:50:55.937685 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34073
--builtin_ntp_servers=127.10.236.212:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 13:50:55.939126 20940 ts_manager.cc:194] Registered new tserver with Master: ef10046c9b194ce88f666e249663bff5 (127.10.236.193:34509)
W20260926 13:50:56.052743 21132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:56.053073 21132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:56.053151 21132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:56.057684 21132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:56.057793 21132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:56.058001 21132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:56.063544 21132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:56.064762 21132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:56.066272 21132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:56.076243 21137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:56.076236 21140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:56.076318 21132 server_base.cc:1034] running on GCE node
W20260926 13:50:56.076272 21138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:56.076920 21132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:56.077531 21132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:56.078703 21132 hybrid_clock.cc:648] HybridClock initialized: now 1790430656078672 us; error 31 us; skew 500 ppm
I20260926 13:50:56.080543 21132 webserver.cc:460] Webserver started at http://127.10.236.194:46009/ using document root <none> and password file <none>
I20260926 13:50:56.081208 21132 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:56.081269 21132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:56.081502 21132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:56.083633 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "a8621ff0d6134e17a48634e14328f025"
format_stamp: "Formatted at 2026-09-26 13:50:56 on dist-test-slave-r34j"
server_key: "c4788b0d7a710bbd428bb8386044d7ef"
server_key_iv: "02d907a2950293d1d9f05a5fd3dd56fe"
server_key_version: "encryptionkey@0"
I20260926 13:50:56.084196 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "a8621ff0d6134e17a48634e14328f025"
format_stamp: "Formatted at 2026-09-26 13:50:56 on dist-test-slave-r34j"
server_key: "c4788b0d7a710bbd428bb8386044d7ef"
server_key_iv: "02d907a2950293d1d9f05a5fd3dd56fe"
server_key_version: "encryptionkey@0"
I20260926 13:50:56.087949 21132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 13:50:56.090507 21146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:56.092051 21132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:56.092178 21132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "a8621ff0d6134e17a48634e14328f025"
format_stamp: "Formatted at 2026-09-26 13:50:56 on dist-test-slave-r34j"
server_key: "c4788b0d7a710bbd428bb8386044d7ef"
server_key_iv: "02d907a2950293d1d9f05a5fd3dd56fe"
server_key_version: "encryptionkey@0"
I20260926 13:50:56.092314 21132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:56.097302 21132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:56.097986 21132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:56.098177 21132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:56.098763 21132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:56.099802 21132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:56.099869 21132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:56.099977 21132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:56.100019 21132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:56.109341 21132 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:45469
I20260926 13:50:56.109362 21259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:45469 every 8 connection(s)
I20260926 13:50:56.110323 21132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:56.114235 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21132
I20260926 13:50:56.114369 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:56.114636 11187 external_mini_cluster.cc:1428] Setting key ee52a127505b219768a192124a6efdc5
I20260926 13:50:56.116863 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34073
--builtin_ntp_servers=127.10.236.212:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 13:50:56.116961 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:56.112248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:55111 (local address 127.10.236.254:34073)
0926 13:50:56.112390 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:56.112394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:56.113534 (+ 1140us) server_negotiation.cc:407] Connection header received
0926 13:50:56.115076 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:56.115079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:56.115127 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:56.115207 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:56.116469 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:56.116472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:56.116475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:56.116488 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:56.116512 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:56.116522 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:56.116615 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:56.116755 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 13:50:56.116802 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:50:56.117919 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:56.112660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34073 (local address 127.10.236.194:55111)
0926 13:50:56.113410 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:56.113450 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:56.114858 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:56.115403 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:56.115412 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:56.115559 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:56.116184 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:56.116187 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:56.116240 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:56.116317 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:56.116616 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:56.116622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:56.116626 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:56.116757 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 13:50:56.117138 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":136,"threads_started":1}
I20260926 13:50:56.119045 21260 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34073
I20260926 13:50:56.119251 21260 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:56.119620 21260 heartbeater.cc:511] Master 127.10.236.254:34073 requested a full tablet report, sending...
I20260926 13:50:56.120534 20940 ts_manager.cc:194] Registered new tserver with Master: a8621ff0d6134e17a48634e14328f025 (127.10.236.194:45469)
W20260926 13:50:56.229610 21263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:56.229863 21263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:56.229931 21263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:56.233973 21263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:56.234026 21263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:56.234120 21263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:56.238960 21263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:34073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:56.240082 21263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:56.241423 21263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:56.251210 21268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:56.251255 21269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:56.251257 21271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:56.251367 21263 server_base.cc:1034] running on GCE node
I20260926 13:50:56.251959 21263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:56.252607 21263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:56.253782 21263 hybrid_clock.cc:648] HybridClock initialized: now 1790430656253744 us; error 43 us; skew 500 ppm
I20260926 13:50:56.255671 21263 webserver.cc:460] Webserver started at http://127.10.236.195:35733/ using document root <none> and password file <none>
I20260926 13:50:56.256214 21263 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:56.256273 21263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:56.256444 21263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:56.258518 21263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "69bc718b0f244d5880f6e45dea8d8185"
format_stamp: "Formatted at 2026-09-26 13:50:56 on dist-test-slave-r34j"
server_key: "2a4a7d005b311cb929f5a2dd8a471b59"
server_key_iv: "ef49a4477043357c77c36d74686723fc"
server_key_version: "encryptionkey@0"
I20260926 13:50:56.259042 21263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "69bc718b0f244d5880f6e45dea8d8185"
format_stamp: "Formatted at 2026-09-26 13:50:56 on dist-test-slave-r34j"
server_key: "2a4a7d005b311cb929f5a2dd8a471b59"
server_key_iv: "ef49a4477043357c77c36d74686723fc"
server_key_version: "encryptionkey@0"
I20260926 13:50:56.262679 21263 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 13:50:56.265293 21277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:56.266520 21263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:56.266633 21263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "69bc718b0f244d5880f6e45dea8d8185"
format_stamp: "Formatted at 2026-09-26 13:50:56 on dist-test-slave-r34j"
server_key: "2a4a7d005b311cb929f5a2dd8a471b59"
server_key_iv: "ef49a4477043357c77c36d74686723fc"
server_key_version: "encryptionkey@0"
I20260926 13:50:56.266729 21263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:56.271236 21263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:56.271869 21263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:56.272009 21263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:56.272490 21263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:56.273394 21263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:56.273465 21263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:56.273532 21263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:56.273582 21263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:56.282429 21263 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:46255
I20260926 13:50:56.282449 21390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:46255 every 8 connection(s)
I20260926 13:50:56.283416 21263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:56.284039 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21263
I20260926 13:50:56.284154 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:56.284511 11187 external_mini_cluster.cc:1428] Setting key 0060572a711b369303df88f7a06d3173
I20260926 13:50:56.290539 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:56.285619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:40825 (local address 127.10.236.254:34073)
0926 13:50:56.285787 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:56.285792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:56.287186 (+ 1394us) server_negotiation.cc:407] Connection header received
0926 13:50:56.288661 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:56.288664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:56.288701 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:56.288774 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:56.290085 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:56.290088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:56.290091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:50:56.290101 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:56.290122 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:50:56.290130 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:56.290208 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:56.290380 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 13:50:56.290432 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:50:56.291584 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:56.286237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:34073 (local address 127.10.236.195:40825)
0926 13:50:56.287024 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:56.287083 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:56.288460 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:56.289025 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:56.289032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:56.289129 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:56.289791 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:50:56.289793 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:56.289837 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:50:56.289931 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:56.290213 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:56.290220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:56.290224 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:50:56.290367 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 13:50:56.290574 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":136,"threads_started":1}
I20260926 13:50:56.292726 21391 heartbeater.cc:347] Connected to a master server at 127.10.236.254:34073
I20260926 13:50:56.292945 21391 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:56.293310 21391 heartbeater.cc:511] Master 127.10.236.254:34073 requested a full tablet report, sending...
I20260926 13:50:56.294291 20940 ts_manager.cc:194] Registered new tserver with Master: 69bc718b0f244d5880f6e45dea8d8185 (127.10.236.195:46255)
I20260926 13:50:56.298440 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 13:50:56.301167 20980 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:50:56.300409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.10.236.254:34073)
0926 13:50:56.300524 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:56.300528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:56.300681 (+ 153us) server_negotiation.cc:407] Connection header received
0926 13:50:56.300778 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:56.300780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:56.300810 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:56.300904 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:56.301075 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38778: BlockingRecv error: recv got EOF from 127.0.0.1:38778 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:50:56.301440 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21001
I20260926 13:50:56.307888 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21132
I20260926 13:50:56.313555 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21263
I20260926 13:50:56.318856 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 20910
2026-09-26T13:50:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (827 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:56 dist-test-slave-r34j krb5kdc[21401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:56 dist-test-slave-r34j krb5kdc[21401](info): set up 2 sockets
Sep 26 13:50:56 dist-test-slave-r34j krb5kdc[21401](info): commencing operation
krb5kdc: starting...
W20260926 13:50:57.376525 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:57 dist-test-slave-r34j krb5kdc[21401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:50:57.504304 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:36171
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:36171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:57.614189 21414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:57.614461 21414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:57.614502 21414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:57.618528 21414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:50:57.618583 21414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:57.618602 21414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:57.618619 21414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:50:57.618662 21414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:50:57.623986 21414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:36171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:36171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:57.625106 21414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:57.626111 21414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:57.634665 21420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:57.634654 21422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:57.634727 21414 server_base.cc:1034] running on GCE node
W20260926 13:50:57.634654 21419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:57.635546 21414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:57.636963 21414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:57.638176 21414 hybrid_clock.cc:648] HybridClock initialized: now 1790430657638147 us; error 47 us; skew 500 ppm
Sep 26 13:50:57 dist-test-slave-r34j krb5kdc[21401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430657, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:57.641075 21414 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:50:57.641997 21414 webserver.cc:460] Webserver started at http://127.10.236.254:45415/ using document root <none> and password file <none>
I20260926 13:50:57.642557 21414 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:57.642642 21414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:57.642876 21414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:57.645030 21414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "d355fbc6767b41b0a29c572d79d973b0"
format_stamp: "Formatted at 2026-09-26 13:50:57 on dist-test-slave-r34j"
server_key: "7005a96a7fa8cc57a77e8c704553ef5b"
server_key_iv: "bb0367f76425d5efc1153d7cc72daa24"
server_key_version: "encryptionkey@0"
I20260926 13:50:57.645597 21414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "d355fbc6767b41b0a29c572d79d973b0"
format_stamp: "Formatted at 2026-09-26 13:50:57 on dist-test-slave-r34j"
server_key: "7005a96a7fa8cc57a77e8c704553ef5b"
server_key_iv: "bb0367f76425d5efc1153d7cc72daa24"
server_key_version: "encryptionkey@0"
I20260926 13:50:57.649406 21414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:50:57.651814 21429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:57.653255 21414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 13:50:57.653403 21414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "d355fbc6767b41b0a29c572d79d973b0"
format_stamp: "Formatted at 2026-09-26 13:50:57 on dist-test-slave-r34j"
server_key: "7005a96a7fa8cc57a77e8c704553ef5b"
server_key_iv: "bb0367f76425d5efc1153d7cc72daa24"
server_key_version: "encryptionkey@0"
I20260926 13:50:57.653513 21414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:57.667790 21414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:57.670542 21414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:57.670742 21414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:57.677711 21414 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:36171
I20260926 13:50:57.678007 21481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:36171 every 8 connection(s)
I20260926 13:50:57.678748 21414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:50:57.680477 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21414
I20260926 13:50:57.680603 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:50:57.680891 11187 external_mini_cluster.cc:1428] Setting key 5a2f83405582e67d8d54a65a6f79c571
I20260926 13:50:57.681869 21482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:57.687340 21482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0: Bootstrap starting.
Sep 26 13:50:57 dist-test-slave-r34j krb5kdc[21401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:57.689674 21482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:57.690572 21482 log.cc:824] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:57.692866 21482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0: No bootstrap required, opened a new log
I20260926 13:50:57.695029 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:57.682371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.10.236.254:36171)
0926 13:50:57.682820 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:57.682835 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:57.682872 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:50:57.683552 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:57.683569 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:57.683894 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:57.684219 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:57.685014 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:57.685992 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:57.686839 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:57.687301 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:57.687466 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:57.689990 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:57.689996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:57.690012 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:57.690042 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:57.692059 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:57.692477 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:57.692483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:57.692489 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:57.692621 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:57.692884 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:57.692888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:57.692890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:57.693372 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:57.693550 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:57.693785 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 13:50:57.694082 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260926 13:50:57.697197 21482 raft_consensus.cc:348] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d355fbc6767b41b0a29c572d79d973b0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36171 } }
I20260926 13:50:57.697469 21482 raft_consensus.cc:374] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:57.697508 21482 raft_consensus.cc:729] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d355fbc6767b41b0a29c572d79d973b0, State: Initialized, Role: FOLLOWER
I20260926 13:50:57.697894 21482 consensus_queue.cc:260] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d355fbc6767b41b0a29c572d79d973b0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36171 } }
I20260926 13:50:57.698007 21482 raft_consensus.cc:388] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:50:57.698046 21482 raft_consensus.cc:482] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:50:57.698114 21482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:57.698966 21482 raft_consensus.cc:504] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d355fbc6767b41b0a29c572d79d973b0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36171 } }
I20260926 13:50:57.699249 21482 leader_election.cc:302] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d355fbc6767b41b0a29c572d79d973b0; no voters:
I20260926 13:50:57.699553 21482 leader_election.cc:288] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:50:57.699657 21487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:57.699892 21487 raft_consensus.cc:686] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [term 1 LEADER]: Becoming Leader. State: Replica: d355fbc6767b41b0a29c572d79d973b0, State: Running, Role: LEADER
I20260926 13:50:57.700220 21487 consensus_queue.cc:237] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d355fbc6767b41b0a29c572d79d973b0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36171 } }
I20260926 13:50:57.700796 21482 sys_catalog.cc:564] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:50:57.701835 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d355fbc6767b41b0a29c572d79d973b0. Latest consensus state: current_term: 1 leader_uuid: "d355fbc6767b41b0a29c572d79d973b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d355fbc6767b41b0a29c572d79d973b0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36171 } } }
I20260926 13:50:57.702039 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:57.702081 21488 sys_catalog.cc:455] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d355fbc6767b41b0a29c572d79d973b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d355fbc6767b41b0a29c572d79d973b0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 36171 } } }
I20260926 13:50:57.702186 21488 sys_catalog.cc:458] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0 [sys.catalog]: This master's current role is: LEADER
I20260926 13:50:57.702503 21496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:50:57.705695 21496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:50:57.711203 21496 catalog_manager.cc:1347] Generated new cluster ID: 2a0604c8a12a46d4b34df18e404e20ab
I20260926 13:50:57.711280 21496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:50:57.729810 21496 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:50:57.731063 21496 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:50:57.738597 21496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0: Generated new TSK 0
I20260926 13:50:57.739449 21496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:50:57.771881 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36171
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:57.886292 21508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:57.886595 21508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:57.886639 21508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:57.890818 21508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:57.890887 21508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:57.890947 21508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:50:57.896317 21508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:57.897444 21508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:57.898442 21508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:57.908447 21516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:57.908449 21514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:57.908392 21513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:57.908576 21508 server_base.cc:1034] running on GCE node
I20260926 13:50:57.909165 21508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:57.909873 21508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:57.911039 21508 hybrid_clock.cc:648] HybridClock initialized: now 1790430657911022 us; error 29 us; skew 500 ppm
Sep 26 13:50:57 dist-test-slave-r34j krb5kdc[21401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430657, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:57.913908 21508 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:50:57.914803 21508 webserver.cc:460] Webserver started at http://127.10.236.193:39135/ using document root <none> and password file <none>
I20260926 13:50:57.915369 21508 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:57.915455 21508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:57.915712 21508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:57.917690 21508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "dd147be9d291463c8f6a37fd0db3d9b6"
format_stamp: "Formatted at 2026-09-26 13:50:57 on dist-test-slave-r34j"
server_key: "73bcbef82111ec95acbd14c1afb57eba"
server_key_iv: "2164c07db0921b53aa7cf2c3e12a47ef"
server_key_version: "encryptionkey@0"
I20260926 13:50:57.918188 21508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "dd147be9d291463c8f6a37fd0db3d9b6"
format_stamp: "Formatted at 2026-09-26 13:50:57 on dist-test-slave-r34j"
server_key: "73bcbef82111ec95acbd14c1afb57eba"
server_key_iv: "2164c07db0921b53aa7cf2c3e12a47ef"
server_key_version: "encryptionkey@0"
I20260926 13:50:57.921864 21508 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:50:57.924440 21523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:57.925791 21508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:50:57.925930 21508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "dd147be9d291463c8f6a37fd0db3d9b6"
format_stamp: "Formatted at 2026-09-26 13:50:57 on dist-test-slave-r34j"
server_key: "73bcbef82111ec95acbd14c1afb57eba"
server_key_iv: "2164c07db0921b53aa7cf2c3e12a47ef"
server_key_version: "encryptionkey@0"
I20260926 13:50:57.926040 21508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:57.948711 21508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:57.951977 21508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:57.952185 21508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:57.952765 21508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:57.953599 21508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:57.953653 21508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:57.953717 21508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:57.953749 21508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:57.962846 21508 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:40909
I20260926 13:50:57.962913 21636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:40909 every 8 connection(s)
I20260926 13:50:57.963847 21508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:50:57.968564 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21508
I20260926 13:50:57.968689 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:50:57.969091 11187 external_mini_cluster.cc:1428] Setting key 599694d20b3bc6bf86973eeb859f5490
Sep 26 13:50:57 dist-test-slave-r34j krb5kdc[21401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430657, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:50:57.978233 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:57.965762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:54645 (local address 127.10.236.254:36171)
0926 13:50:57.965914 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:57.965918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:57.966769 (+ 851us) server_negotiation.cc:407] Connection header received
0926 13:50:57.968238 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:57.968241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:57.968326 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:57.968428 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:57.969795 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:57.970542 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:57.971726 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:57.971971 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:57.972077 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:57.974602 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:57.974605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:57.974608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:57.974632 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:57.976138 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:57.976679 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:57.976683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:57.976685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:57.976768 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:57.977122 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:57.977127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:57.977129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:57.977311 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:57.977424 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:57.977926 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 13:50:57.978064 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 13:50:57.979202 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:57.966146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36171 (local address 127.10.236.193:54645)
0926 13:50:57.966616 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:57.966659 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:57.968028 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:57.968598 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:57.968606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:57.969040 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:57.969650 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:57.969675 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:57.970653 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:57.970656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:57.971610 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:57.971619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:57.972125 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:57.972129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:57.972298 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:57.972951 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:57.972967 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:57.974423 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:57.976268 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:57.976274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:57.976292 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:57.976573 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:57.976867 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:57.976870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:57.976872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:57.977029 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:57.977415 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:57.977422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:57.977688 (+ 266us) client_negotiation.cc:769] Sending connection context
0926 13:50:57.977890 (+ 202us) client_negotiation.cc:240] Negotiation successful
0926 13:50:57.978241 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":133,"threads_started":1}
I20260926 13:50:57.980525 21637 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:50:57.980806 21637 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:57.981355 21637 heartbeater.cc:511] Master 127.10.236.254:36171 requested a full tablet report, sending...
I20260926 13:50:57.982865 21446 ts_manager.cc:194] Registered new tserver with Master: dd147be9d291463c8f6a37fd0db3d9b6 (127.10.236.193:40909)
I20260926 13:50:57.984058 21446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:54645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:50:57.998842 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36171
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:58.111297 21642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:58.111608 21642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:58.111685 21642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:58.115890 21642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:58.115943 21642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:58.116046 21642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:50:58.121249 21642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:58.122316 21642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:58.123322 21642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:50:58.133152 21642 server_base.cc:1034] running on GCE node
W20260926 13:50:58.133082 21647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:58.133104 21648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:58.133076 21650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:58.133793 21642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:58.134442 21642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:58.135614 21642 hybrid_clock.cc:648] HybridClock initialized: now 1790430658135583 us; error 40 us; skew 500 ppm
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430658, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:58.138554 21642 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:50:58.139449 21642 webserver.cc:460] Webserver started at http://127.10.236.194:45753/ using document root <none> and password file <none>
I20260926 13:50:58.140205 21642 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:58.140410 21642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:58.140646 21642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:58.142649 21642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "fea390be21054347a2ac6a22f610442a"
format_stamp: "Formatted at 2026-09-26 13:50:58 on dist-test-slave-r34j"
server_key: "dac4c8e2d2f0fb6b0282824866d91070"
server_key_iv: "98c2f604d1daf3033e5c8ee9e25f0631"
server_key_version: "encryptionkey@0"
I20260926 13:50:58.143231 21642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "fea390be21054347a2ac6a22f610442a"
format_stamp: "Formatted at 2026-09-26 13:50:58 on dist-test-slave-r34j"
server_key: "dac4c8e2d2f0fb6b0282824866d91070"
server_key_iv: "98c2f604d1daf3033e5c8ee9e25f0631"
server_key_version: "encryptionkey@0"
I20260926 13:50:58.147054 21642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:50:58.149659 21657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:58.150990 21642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:50:58.151132 21642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "fea390be21054347a2ac6a22f610442a"
format_stamp: "Formatted at 2026-09-26 13:50:58 on dist-test-slave-r34j"
server_key: "dac4c8e2d2f0fb6b0282824866d91070"
server_key_iv: "98c2f604d1daf3033e5c8ee9e25f0631"
server_key_version: "encryptionkey@0"
I20260926 13:50:58.151245 21642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:58.183892 21642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:58.187460 21642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:58.187752 21642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:58.188383 21642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:58.189287 21642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:58.189337 21642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:58.189436 21642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:58.189476 21642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:58.198573 21642 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:39667
I20260926 13:50:58.198668 21770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:39667 every 8 connection(s)
I20260926 13:50:58.199602 21642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:50:58.206022 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21642
I20260926 13:50:58.206159 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:50:58.206429 11187 external_mini_cluster.cc:1428] Setting key f0eee2c8f8dad14128a8a8624cf33a5a
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430658, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:58.214591 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.201698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:45467 (local address 127.10.236.254:36171)
0926 13:50:58.201828 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.201832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.202691 (+ 859us) server_negotiation.cc:407] Connection header received
0926 13:50:58.204189 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.204192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.204248 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.204328 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:58.205482 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.206030 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.207469 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.207882 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.207997 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.210491 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:58.210497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:58.210503 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:58.210536 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:58.212395 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:58.212932 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:58.212938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:58.212943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:58.213050 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:58.213368 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:58.213374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:58.213378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:58.213652 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:58.213774 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.214265 (+ 491us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.214404 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 13:50:58.215428 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.202089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36171 (local address 127.10.236.194:45467)
0926 13:50:58.202548 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.202589 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.203984 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.204501 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.204508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.204859 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:58.205339 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.205352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.206178 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.206182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.207352 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.207362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.208026 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.208029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.208170 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.208743 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:58.208759 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:58.210286 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:58.212513 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:58.212518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:58.212532 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:58.212791 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:58.213145 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:58.213147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:58.213149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:58.213276 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:58.213756 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:58.213761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:58.214021 (+ 260us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.214180 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.214429 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:50:58.216754 21771 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36171
I20260926 13:50:58.216974 21771 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:58.217514 21771 heartbeater.cc:511] Master 127.10.236.254:36171 requested a full tablet report, sending...
I20260926 13:50:58.218580 21446 ts_manager.cc:194] Registered new tserver with Master: fea390be21054347a2ac6a22f610442a (127.10.236.194:39667)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:50:58.219214 21446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:45467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:50:58.239068 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36171
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:50:58.352777 21776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:50:58.353060 21776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:50:58.353130 21776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:50:58.357317 21776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:50:58.357393 21776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:50:58.357479 21776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:50:58.362686 21776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:36171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:50:58.363801 21776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:50:58.364826 21776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:50:58.374217 21781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:58.374271 21782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:50:58.374234 21784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:50:58.374308 21776 server_base.cc:1034] running on GCE node
I20260926 13:50:58.374931 21776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:50:58.375600 21776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:50:58.376781 21776 hybrid_clock.cc:648] HybridClock initialized: now 1790430658376768 us; error 34 us; skew 500 ppm
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430658, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:50:58.379588 21776 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:50:58.380507 21776 webserver.cc:460] Webserver started at http://127.10.236.195:45403/ using document root <none> and password file <none>
I20260926 13:50:58.381062 21776 fs_manager.cc:359] Metadata directory not provided
I20260926 13:50:58.381146 21776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:50:58.381374 21776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:50:58.383397 21776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "3399805072214fe8894dfc84b0d8baa1"
format_stamp: "Formatted at 2026-09-26 13:50:58 on dist-test-slave-r34j"
server_key: "680cfe24d671888076aa04c4202cc86b"
server_key_iv: "cdbd9cf329ebd286ed29de03546f5001"
server_key_version: "encryptionkey@0"
I20260926 13:50:58.384034 21776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "3399805072214fe8894dfc84b0d8baa1"
format_stamp: "Formatted at 2026-09-26 13:50:58 on dist-test-slave-r34j"
server_key: "680cfe24d671888076aa04c4202cc86b"
server_key_iv: "cdbd9cf329ebd286ed29de03546f5001"
server_key_version: "encryptionkey@0"
I20260926 13:50:58.387921 21776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:50:58.390331 21791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:58.391671 21776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:50:58.391806 21776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "3399805072214fe8894dfc84b0d8baa1"
format_stamp: "Formatted at 2026-09-26 13:50:58 on dist-test-slave-r34j"
server_key: "680cfe24d671888076aa04c4202cc86b"
server_key_iv: "cdbd9cf329ebd286ed29de03546f5001"
server_key_version: "encryptionkey@0"
I20260926 13:50:58.391911 21776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:50:58.415165 21776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:50:58.418617 21776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:50:58.418825 21776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:50:58.419436 21776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:50:58.420377 21776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:50:58.420444 21776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:58.420512 21776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:50:58.420557 21776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:50:58.429939 21776 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:37663
I20260926 13:50:58.429963 21904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:37663 every 8 connection(s)
I20260926 13:50:58.430907 21776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:50:58.436033 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 21776
I20260926 13:50:58.436138 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:50:58.436462 11187 external_mini_cluster.cc:1428] Setting key 4226d40efc5ba2aa5c802eee0a06e241
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430658, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:50:58.446075 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.432975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:36257 (local address 127.10.236.254:36171)
0926 13:50:58.433138 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.433143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.434049 (+ 906us) server_negotiation.cc:407] Connection header received
0926 13:50:58.435408 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.435412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.435463 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.435562 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:58.437022 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.437742 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.439125 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.439369 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.439482 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.442159 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:58.442163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:58.442166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:58.442201 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:58.444022 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:58.444563 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:58.444568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:58.444570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:58.444652 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:58.444995 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:58.445000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:58.445002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:58.445181 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:58.445301 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.445790 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.445915 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 13:50:58.446888 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.433359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:36171 (local address 127.10.236.195:36257)
0926 13:50:58.433893 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.433937 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.435197 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.435804 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.435812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.436309 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:50:58.436868 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.436884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.437867 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.437872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.439003 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.439012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.439552 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.439557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.439733 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.440302 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:50:58.440319 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:50:58.441971 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:50:58.444147 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:58.444153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:58.444170 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:58.444457 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:58.444763 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:50:58.444770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:50:58.444772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:50:58.444903 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:50:58.445311 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:50:58.445319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:50:58.445560 (+ 241us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.445758 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.446081 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":110,"threads_started":1}
I20260926 13:50:58.448047 21905 heartbeater.cc:347] Connected to a master server at 127.10.236.254:36171
I20260926 13:50:58.448298 21905 heartbeater.cc:464] Registering TS with master...
I20260926 13:50:58.448772 21905 heartbeater.cc:511] Master 127.10.236.254:36171 requested a full tablet report, sending...
I20260926 13:50:58.449940 21446 ts_manager.cc:194] Registered new tserver with Master: 3399805072214fe8894dfc84b0d8baa1 (127.10.236.195:37663)
I20260926 13:50:58.450493 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:50:58.450698 21446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:36257
I20260926 13:50:58.459666 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.452553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.10.236.254:36171)
0926 13:50:58.452662 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.452666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.452853 (+ 187us) server_negotiation.cc:407] Connection header received
0926 13:50:58.453009 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.453012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.453057 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.453132 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:50:58.453793 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.454335 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.455157 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.455397 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.455519 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.456803 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:50:58.456805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:50:58.456807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:50:58.456843 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:50:58.458299 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:58.458650 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:58.458652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:58.458654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:58.458720 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:50:58.458955 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:50:58.458957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:50:58.458958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:50:58.459142 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:50:58.459241 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.459429 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.459562 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:50:58.462357 21446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:50:58.465628 21446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:50:58.480451 21917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.474145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40909 (local address 127.0.0.1:50270)
0926 13:50:58.475265 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.475279 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.475387 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.475921 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.475936 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.475966 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:58.476094 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.476101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.477934 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.477938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.478805 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.478814 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.479848 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.479852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.479970 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.480012 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.480134 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.480258 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"thread_start_us":81,"threads_started":1}
I20260926 13:50:58.480482 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.474381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.10.236.193:40909)
0926 13:50:58.475204 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.475224 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.475379 (+ 155us) server_negotiation.cc:407] Connection header received
0926 13:50:58.475524 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.475530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.475757 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.475912 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:58.476209 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.477785 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.478937 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.479656 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.479845 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.479991 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.480191 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.480318 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":122,"threads_started":1}
I20260926 13:50:58.481221 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.474114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:39667 (local address 127.0.0.1:53444)
0926 13:50:58.474870 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.474935 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.475057 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.476333 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.476336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.476353 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:58.476451 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.476459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.477784 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.477788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.479248 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.479259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.480410 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.480413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.480514 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.480541 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.480602 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.480680 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":123,"thread_start_us":76,"threads_started":1}
I20260926 13:50:58.481259 21919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.474837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37663 (local address 127.0.0.1:32792)
0926 13:50:58.475596 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.475605 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.475672 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.476261 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.476264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.476281 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:58.476362 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.476368 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.477700 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.477703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.479248 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.479259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.480778 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.480782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.480883 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.480907 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.481030 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.481107 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":115,"thread_start_us":81,"threads_started":1}
I20260926 13:50:58.481690 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.475205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.10.236.195:37663)
0926 13:50:58.475567 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.475579 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.475657 (+ 78us) server_negotiation.cc:407] Connection header received
0926 13:50:58.475758 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.475763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.475954 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.476090 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:58.476497 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.477558 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.479382 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.480227 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.480358 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.480903 (+ 545us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.481407 (+ 504us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.481512 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":136,"threads_started":1}
I20260926 13:50:58.481997 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.474915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.10.236.194:39667)
0926 13:50:58.475908 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.475919 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.475940 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:50:58.476004 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.476009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.476199 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.476343 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:58.476568 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.477600 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.479392 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.480270 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.481574 (+ 1304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.481699 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.481749 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.481841 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":100,"threads_started":1}
I20260926 13:50:58.484125 21571 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf784561c59429fb38ac36f6058ee1a (DEFAULT_TABLE table=test-table [id=354df6a7e207491eaac6a485c5dd58ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:58.484115 21839 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf784561c59429fb38ac36f6058ee1a (DEFAULT_TABLE table=test-table [id=354df6a7e207491eaac6a485c5dd58ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:58.484550 21705 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf784561c59429fb38ac36f6058ee1a (DEFAULT_TABLE table=test-table [id=354df6a7e207491eaac6a485c5dd58ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:50:58.485296 21571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf784561c59429fb38ac36f6058ee1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:58.485301 21839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf784561c59429fb38ac36f6058ee1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:58.485364 21705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf784561c59429fb38ac36f6058ee1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:50:58.491834 21922 tablet_bootstrap.cc:492] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Bootstrap starting.
I20260926 13:50:58.493086 21923 tablet_bootstrap.cc:492] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: Bootstrap starting.
I20260926 13:50:58.494134 21922 tablet_bootstrap.cc:654] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:58.494927 21923 tablet_bootstrap.cc:654] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:58.495357 21924 tablet_bootstrap.cc:492] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Bootstrap starting.
I20260926 13:50:58.495383 21922 log.cc:824] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:58.496680 21923 log.cc:824] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:58.497645 21922 tablet_bootstrap.cc:492] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: No bootstrap required, opened a new log
I20260926 13:50:58.497853 21922 ts_tablet_manager.cc:1397] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:50:58.498181 21924 tablet_bootstrap.cc:654] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Neither blocks nor log segments found. Creating new log.
I20260926 13:50:58.498700 21923 tablet_bootstrap.cc:492] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: No bootstrap required, opened a new log
I20260926 13:50:58.498886 21923 ts_tablet_manager.cc:1397] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260926 13:50:58.499007 21924 log.cc:824] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:50:58.501019 21924 tablet_bootstrap.cc:492] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: No bootstrap required, opened a new log
I20260926 13:50:58.501219 21924 ts_tablet_manager.cc:1397] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:50:58.501927 21922 raft_consensus.cc:348] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.502256 21922 raft_consensus.cc:374] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:58.502336 21922 raft_consensus.cc:729] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3399805072214fe8894dfc84b0d8baa1, State: Initialized, Role: FOLLOWER
I20260926 13:50:58.502861 21922 consensus_queue.cc:260] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.503261 21923 raft_consensus.cc:348] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.503521 21923 raft_consensus.cc:374] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:58.503607 21923 raft_consensus.cc:729] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd147be9d291463c8f6a37fd0db3d9b6, State: Initialized, Role: FOLLOWER
I20260926 13:50:58.503973 21922 ts_tablet_manager.cc:1428] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:50:58.504034 21923 consensus_queue.cc:260] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.504329 21905 heartbeater.cc:503] Master 127.10.236.254:36171 was elected leader, sending a full tablet report...
I20260926 13:50:58.505024 21924 raft_consensus.cc:348] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.505359 21924 raft_consensus.cc:374] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:50:58.505457 21924 raft_consensus.cc:729] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea390be21054347a2ac6a22f610442a, State: Initialized, Role: FOLLOWER
I20260926 13:50:58.505870 21924 consensus_queue.cc:260] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.505944 21923 ts_tablet_manager.cc:1428] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260926 13:50:58.506281 21637 heartbeater.cc:503] Master 127.10.236.254:36171 was elected leader, sending a full tablet report...
I20260926 13:50:58.506783 21924 ts_tablet_manager.cc:1428] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 13:50:58.506893 21771 heartbeater.cc:503] Master 127.10.236.254:36171 was elected leader, sending a full tablet report...
W20260926 13:50:58.681949 21906 tablet.cc:2369] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:58.696651 21930 raft_consensus.cc:482] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:50:58.696913 21930 raft_consensus.cc:504] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.698081 21930 leader_election.cc:288] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd147be9d291463c8f6a37fd0db3d9b6 (127.10.236.193:40909), 3399805072214fe8894dfc84b0d8baa1 (127.10.236.195:37663)
W20260926 13:50:58.700722 21772 tablet.cc:2369] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:58.703538 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.698472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:43115 (local address 127.10.236.193:40909)
0926 13:50:58.698603 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.698608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.698949 (+ 341us) server_negotiation.cc:407] Connection header received
0926 13:50:58.699015 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.699018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.699055 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.699114 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:58.699362 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.700129 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.702102 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.702799 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.702941 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.702988 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.703320 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.703391 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 13:50:58.703538 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.698490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:56085 (local address 127.10.236.195:37663)
0926 13:50:58.698639 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.698644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.698662 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:50:58.698726 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.698728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.698771 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.698837 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:50:58.699198 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.700019 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.702109 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.702762 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.702907 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.702962 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.703320 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.703391 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:50:58.703577 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.698356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37663 (local address 127.10.236.194:56085)
0926 13:50:58.698487 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.698504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.698612 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.698876 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.698882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.698907 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:58.699064 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.699070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.700162 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.700165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.701950 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.701962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.702937 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.702941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.703191 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.703209 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.703279 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.703373 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":297,"spinlock_wait_cycles":2816}
I20260926 13:50:58.703585 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.698356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40909 (local address 127.10.236.194:43115)
0926 13:50:58.698832 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.698849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:50:58.698949 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:50:58.699152 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:50:58.699155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:50:58.699174 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:50:58.699268 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.699273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.700240 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.700243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.701954 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:50:58.701962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.702937 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.702941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:50:58.703161 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.703191 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 13:50:58.703279 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:50:58.703378 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":337,"thread_start_us":85,"threads_started":1}
I20260926 13:50:58.704269 21859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf784561c59429fb38ac36f6058ee1a" candidate_uuid: "fea390be21054347a2ac6a22f610442a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3399805072214fe8894dfc84b0d8baa1" is_pre_election: true
I20260926 13:50:58.704552 21591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf784561c59429fb38ac36f6058ee1a" candidate_uuid: "fea390be21054347a2ac6a22f610442a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" is_pre_election: true
I20260926 13:50:58.704614 21859 raft_consensus.cc:2454] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fea390be21054347a2ac6a22f610442a in term 0.
I20260926 13:50:58.704819 21591 raft_consensus.cc:2454] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fea390be21054347a2ac6a22f610442a in term 0.
I20260926 13:50:58.705076 21659 leader_election.cc:302] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3399805072214fe8894dfc84b0d8baa1, fea390be21054347a2ac6a22f610442a; no voters:
I20260926 13:50:58.705386 21930 raft_consensus.cc:2781] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:50:58.705472 21930 raft_consensus.cc:482] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:50:58.705523 21930 raft_consensus.cc:3037] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:58.706557 21930 raft_consensus.cc:504] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.707068 21930 leader_election.cc:288] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [CANDIDATE]: Term 1 election: Requested vote from peers dd147be9d291463c8f6a37fd0db3d9b6 (127.10.236.193:40909), 3399805072214fe8894dfc84b0d8baa1 (127.10.236.195:37663)
I20260926 13:50:58.707515 21859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf784561c59429fb38ac36f6058ee1a" candidate_uuid: "fea390be21054347a2ac6a22f610442a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3399805072214fe8894dfc84b0d8baa1"
I20260926 13:50:58.707532 21591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf784561c59429fb38ac36f6058ee1a" candidate_uuid: "fea390be21054347a2ac6a22f610442a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd147be9d291463c8f6a37fd0db3d9b6"
I20260926 13:50:58.707649 21591 raft_consensus.cc:3037] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:58.707649 21859 raft_consensus.cc:3037] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:50:58.708916 21859 raft_consensus.cc:2454] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fea390be21054347a2ac6a22f610442a in term 1.
I20260926 13:50:58.708916 21591 raft_consensus.cc:2454] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fea390be21054347a2ac6a22f610442a in term 1.
I20260926 13:50:58.709208 21659 leader_election.cc:302] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3399805072214fe8894dfc84b0d8baa1, fea390be21054347a2ac6a22f610442a; no voters:
I20260926 13:50:58.709357 21930 raft_consensus.cc:2781] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:50:58.709553 21930 raft_consensus.cc:686] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 1 LEADER]: Becoming Leader. State: Replica: fea390be21054347a2ac6a22f610442a, State: Running, Role: LEADER
I20260926 13:50:58.709946 21930 consensus_queue.cc:237] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } }
I20260926 13:50:58.713177 21445 catalog_manager.cc:5495] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a reported cstate change: term changed from 0 to 1, leader changed from <none> to fea390be21054347a2ac6a22f610442a (127.10.236.194). New cstate: current_term: 1 leader_uuid: "fea390be21054347a2ac6a22f610442a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea390be21054347a2ac6a22f610442a" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 } health_report { overall_health: UNKNOWN } } }
W20260926 13:50:58.715303 21638 tablet.cc:2369] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:50:58.774734 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.770717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.10.236.194:39667)
0926 13:50:58.770881 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 13:50:58.770884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:50:58.770920 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:50:58.770998 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:50:58.771000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:50:58.771068 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:50:58.771152 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:50:58.771570 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.772173 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.773284 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:50:58.773499 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:50:58.773600 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:50:58.773933 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:50:58.774361 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:50:58.774458 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:50:58.774566 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 13:50:58.774617 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 13:50:58.780743 21859 raft_consensus.cc:1261] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 1 FOLLOWER]: Refusing update from remote peer fea390be21054347a2ac6a22f610442a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:58.780743 21591 raft_consensus.cc:1261] T 9bf784561c59429fb38ac36f6058ee1a P dd147be9d291463c8f6a37fd0db3d9b6 [term 1 FOLLOWER]: Refusing update from remote peer fea390be21054347a2ac6a22f610442a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:50:58.781548 21932 consensus_queue.cc:1035] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd147be9d291463c8f6a37fd0db3d9b6" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:58.781785 21930 consensus_queue.cc:1035] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3399805072214fe8894dfc84b0d8baa1" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:50:58.799435 21445 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:58.799670 21445 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:50:58.802310 21445 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d355fbc6767b41b0a29c572d79d973b0: Sending DeleteTablet for 3 replicas of tablet 9bf784561c59429fb38ac36f6058ee1a
I20260926 13:50:58.803217 21839 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bf784561c59429fb38ac36f6058ee1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:58 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:32792
I20260926 13:50:58.803210 21571 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bf784561c59429fb38ac36f6058ee1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:58 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:50270
I20260926 13:50:58.803360 21705 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bf784561c59429fb38ac36f6058ee1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:50:58 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:53444
I20260926 13:50:58.803934 21947 tablet_replica.cc:330] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: stopping tablet replica
I20260926 13:50:58.803979 21946 tablet_replica.cc:330] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: stopping tablet replica
I20260926 13:50:58.804306 21947 raft_consensus.cc:2229] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:50:58.804422 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21508
I20260926 13:50:58.804731 21947 raft_consensus.cc:2258] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:58.804306 21946 raft_consensus.cc:2229] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:50:58.806411 21947 ts_tablet_manager.cc:1905] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:58.806551 21946 raft_consensus.cc:2258] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:50:58.808166 21946 ts_tablet_manager.cc:1905] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:50:58.811663 21946 ts_tablet_manager.cc:1918] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:58.811834 21946 log.cc:1196] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/9bf784561c59429fb38ac36f6058ee1a
I20260926 13:50:58.812259 21946 ts_tablet_manager.cc:1939] T 9bf784561c59429fb38ac36f6058ee1a P 3399805072214fe8894dfc84b0d8baa1: Deleting consensus metadata
W20260926 13:50:58.812616 21433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:40909 (error 108)
I20260926 13:50:58.813462 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21642
I20260926 13:50:58.813846 21947 ts_tablet_manager.cc:1918] T 9bf784561c59429fb38ac36f6058ee1a P fea390be21054347a2ac6a22f610442a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:50:58.813809 21431 catalog_manager.cc:4841] TS 3399805072214fe8894dfc84b0d8baa1 (127.10.236.195:37663): tablet 9bf784561c59429fb38ac36f6058ee1a (table test-table [id=354df6a7e207491eaac6a485c5dd58ba]) successfully deleted
I20260926 13:50:58.814878 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.814576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40909 (local address 127.0.0.1:50280)
0926 13:50:58.814714 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.814793 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:40909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 13:50:58.815095 21433 catalog_manager.cc:4584] TS dd147be9d291463c8f6a37fd0db3d9b6 (127.10.236.193:40909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9bf784561c59429fb38ac36f6058ee1a: Network error: Client connection negotiation failed: client connection to 127.10.236.193:40909: connect: Connection refused (error 111)
I20260926 13:50:58.819649 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21776
I20260926 13:50:58.820508 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:50:58.819860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:39667 (local address 127.0.0.1:53464)
0926 13:50:58.820315 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 13:50:58.820407 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.194:39667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357}
I20260926 13:50:58.825186 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 21414
2026-09-26T13:50:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2507 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21951](info): set up 2 sockets
Sep 26 13:50:58 dist-test-slave-r34j krb5kdc[21951](info): commencing operation
krb5kdc: starting...
W20260926 13:50:59.864531 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:50:59 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:50:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:50:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/rangeradmin_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/rangeradmin_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/rangerlookup_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/rangerlookup_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
I20260926 13:51:00.046831 11187 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMS3Khi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/postgres -l logfile start
2026-09-26 13:51:02.816 UTC [21981] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 13:51:02.817 UTC [21981] LOG: listening on IPv4 address "127.10.236.212", port 40313
2026-09-26 13:51:02.820 UTC [21981] LOG: listening on Unix socket "/tmp/.s.PGSQL.40313"
2026-09-26 13:51:02.856 UTC [21984] LOG: database system was shut down at 2026-09-26 13:51:01 UTC
2026-09-26 13:51:02.860 UTC [21981] LOG: database system is ready to accept connections
I20260926 13:51:03.812296 11187 mini_postgres.cc:96] Postgres bound to 40313
2026-09-26 13:51:03.818 UTC [21992] FATAL: database "slave" does not exist
127.10.236.212:40313 - accepting connections
I20260926 13:51:03.819639 11187 mini_ranger.cc:161] Starting Ranger...
I20260926 13:51:03.834545 11187 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 13:51:04.412956 11187 mini_ranger.cc:87] Created ranger Postgres database
I20260926 13:51:04.413066 11187 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-admin
2026-09-26 13:51:04,695 [I] DB FLAVOR :POSTGRES
2026-09-26 13:51:04,695 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:51:04,696 [I] Checking connection..
2026-09-26 13:51:04,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 13:51:04,962 [I] Checking connection passed.
2026-09-26 13:51:04,962 [I] --------- Verifying version history table ---------
2026-09-26 13:51:04,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:51:05,204 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:51:05,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:51:05,451 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:51:05,451 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 13:51:05,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 13:51:05.700 UTC [22056] WARNING: there is no transaction in progress
2026-09-26 13:51:05,707 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 13:51:05,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:51:05,977 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 13:51:05,977 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:51:05,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 13:51:06,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 13:51:06,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-r34j.c.gcp-upstream.internal','N') ;"
2026-09-26 13:51:06,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:51:06,978 [I] Table x_portal_user does not exist in database ranger
2026-09-26 13:51:06,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:51:07,220 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 13:51:07,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:51:07,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:51:07,722 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 13:51:07,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 13:51:08.783 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.802 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.821 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.839 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.855 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.874 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.934 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.943 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.962 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.975 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:08.987 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:09.002 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:09.014 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:09.024 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:09.032 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:09.040 UTC [22164] WARNING: there is no transaction in progress
2026-09-26 13:51:09,786 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 13:51:09,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 13:51:10,048 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 13:51:10,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:51:10,282 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 13:51:10,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:51:10,540 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 13:51:10,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:51:10,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:51:11,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-r34j.c.gcp-upstream.internal';"
2026-09-26 13:51:11,311 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 13:51:11,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 13:51:11,582 [I] DB_PATCHES have already been applied
I20260926 13:51:11.588264 11187 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMS3Khi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 13:51:11.588358 11187 mini_ranger.cc:193] Using host: 127.10.236.212
I20260926 13:51:11.592031 11187 mini_ranger.cc:227] Ranger admin URL: http://127.10.236.212:36465
Sep 26, 2026 1:51:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:51:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:51:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 1:51:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 1:51:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 1:51:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 1:51:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 13:51:12.807374 11187 mini_ranger.cc:160] Time spent starting Ranger: real 8.988s user 0.000s sys 0.010s
Sep 26, 2026 1:51:33 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 1:51:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 1:51:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 1:51:33 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 1:51:34 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 13:51:34.911047 11187 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/rangerkms_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/rangerkms_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.236.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.236.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 13:51:34 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430694, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 13:51:35.000324 11187 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 13:51:35.010790 11187 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 13:51:36.080873 11187 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 13:51:36.080957 11187 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms
I20260926 13:51:36.081069 11187 mini_ranger_kms.cc:227] Using postgres at 127.10.236.212:40313
2026-09-26 13:51:36,282 [I] DB FLAVOR :POSTGRES
2026-09-26 13:51:36,282 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:51:36,283 [I] Checking connection
2026-09-26 13:51:36,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 13:51:36,557 [I] connection success
2026-09-26 13:51:36,558 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 13:51:36,558 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 13:51:36,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 13:51:36,795 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 13:51:36,795 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:51:36,796 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 13:51:36,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:40313/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 13:51:37,089 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 13:51:37.572786 11187 mini_ranger_kms.cc:310] Created kms service
I20260926 13:51:37.751542 11187 mini_ranger_kms.cc:326] Created kudu user
I20260926 13:51:37.795593 11187 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 13:51:37 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430694, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:51:38.201478 11187 mini_ranger_kms.cc:384] Added ranger policy
I20260926 13:51:38.201664 11187 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 13:51:38.201700 11187 mini_ranger_kms.cc:240] Using host: 127.10.236.212
I20260926 13:51:38.206159 11187 mini_ranger_kms.cc:276] Ranger KMS PID: 22344
I20260926 13:51:38.206316 11187 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.236.212:49913
Sep 26, 2026 1:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 1:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 1:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 1:51:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 13:51:40.469902 11187 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.470s user 0.018s sys 0.162s
I20260926 13:51:40.470050 11187 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 13:51:40.470183 11187 mini_ranger_kms.cc:401] 127.10.236.212:49913/kms/v1/keys
Sep 26 13:51:40 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790430700, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.236.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 13:51:41 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430700, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.236.212@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
Sep 26 13:51:41 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430700, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.236.212@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
Sep 26 13:51:43 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:51:43.716252 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:38161
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:38161
--ranger_config_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:49913/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:51:43.861301 22395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:51:43.861613 22395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:51:43.861698 22395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:51:43.865810 22395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:51:43.865904 22395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:51:43.865952 22395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:51:43.866010 22395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:51:43.866050 22395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:51:43.871881 22395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.236.212:49913/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:38161
--ranger_config_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:51:43.873168 22395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:51:43.874291 22395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:51:43.883983 22400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:51:43.884130 22401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:51:43.883940 22403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:51:43.884485 22395 server_base.cc:1034] running on GCE node
I20260926 13:51:43.885211 22395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:51:43.886387 22395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:51:43.887569 22395 hybrid_clock.cc:648] HybridClock initialized: now 1790430703887541 us; error 44 us; skew 500 ppm
Sep 26 13:51:43 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430703, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:51:43.894815 22395 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:51:43.895974 22395 webserver.cc:460] Webserver started at http://127.10.236.254:46447/ using document root <none> and password file <none>
I20260926 13:51:43.896541 22395 fs_manager.cc:359] Metadata directory not provided
I20260926 13:51:43.896597 22395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:51:43.896831 22395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:51:43 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430703, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:51:43.953598 22395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "5187d92aec5b4ce3bca0f3603fc0b626"
format_stamp: "Formatted at 2026-09-26 13:51:43 on dist-test-slave-r34j"
server_key: "2675d6453f7ef5871244ca862f290d54"
server_key_iv: "6e523294c3a051ec8a07b2e8065666d2"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:43.954324 22395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "5187d92aec5b4ce3bca0f3603fc0b626"
format_stamp: "Formatted at 2026-09-26 13:51:43 on dist-test-slave-r34j"
server_key: "2675d6453f7ef5871244ca862f290d54"
server_key_iv: "6e523294c3a051ec8a07b2e8065666d2"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:43.959040 22395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:51:43.977101 22410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:43.978797 22395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:51:43.978951 22395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "5187d92aec5b4ce3bca0f3603fc0b626"
format_stamp: "Formatted at 2026-09-26 13:51:43 on dist-test-slave-r34j"
server_key: "2675d6453f7ef5871244ca862f290d54"
server_key_iv: "6e523294c3a051ec8a07b2e8065666d2"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:43.979096 22395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:51:44.007151 22395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:51:44.010964 22395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:51:44.011181 22395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:51:44.020738 22395 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:38161
I20260926 13:51:44.020793 22462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:38161 every 8 connection(s)
I20260926 13:51:44.022037 22395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:51:44.025180 22463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:51:44.025792 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 22395
I20260926 13:51:44.025916 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:51:44.026219 11187 external_mini_cluster.cc:1428] Setting key 0c5ffc6f1554dfad386ee0ac0503277e
I20260926 13:51:44.032379 22463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626: Bootstrap starting.
I20260926 13:51:44.034947 22463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626: Neither blocks nor log segments found. Creating new log.
I20260926 13:51:44.035864 22463 log.cc:824] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:51:44 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:51:44.038108 22463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626: No bootstrap required, opened a new log
I20260926 13:51:44.041834 22463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38161 } }
I20260926 13:51:44.042138 22463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:51:44.042236 22463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5187d92aec5b4ce3bca0f3603fc0b626, State: Initialized, Role: FOLLOWER
I20260926 13:51:44.042769 22463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38161 } }
I20260926 13:51:44.043061 22463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:51:44.043298 22463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:51:44.043457 22463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:51:44.044570 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:44.028221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.10.236.254:38161)
0926 13:51:44.028728 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:44.028747 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:44.028792 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:51:44.029686 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:44.029710 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:44.030162 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:44.030544 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:51:44.031426 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:44.032983 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:44.034184 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:44.034945 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:44.035591 (+ 646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:44.038306 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:51:44.038313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:51:44.038330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:51:44.038362 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:51:44.041160 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:44.041732 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:44.041739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:44.041745 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:44.041884 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:44.042226 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:44.042229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:44.042231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:44.042742 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:51:44.042965 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:44.043276 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 13:51:44.043594 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":151,"threads_started":1}
I20260926 13:51:44.044708 22463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38161 } }
I20260926 13:51:44.045111 22463 leader_election.cc:302] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5187d92aec5b4ce3bca0f3603fc0b626; no voters:
I20260926 13:51:44.045459 22463 leader_election.cc:288] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:51:44.045581 22468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:51:44.045868 22468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [term 1 LEADER]: Becoming Leader. State: Replica: 5187d92aec5b4ce3bca0f3603fc0b626, State: Running, Role: LEADER
I20260926 13:51:44.046316 22468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38161 } }
I20260926 13:51:44.046972 22463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:51:44.047689 22468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38161 } } }
I20260926 13:51:44.047802 22469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5187d92aec5b4ce3bca0f3603fc0b626. Latest consensus state: current_term: 1 leader_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5187d92aec5b4ce3bca0f3603fc0b626" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38161 } } }
I20260926 13:51:44.047902 22468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [sys.catalog]: This master's current role is: LEADER
I20260926 13:51:44.048108 22469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626 [sys.catalog]: This master's current role is: LEADER
I20260926 13:51:44.048592 22477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:51:44.052683 22477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:51:44.053150 22463 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 13:51:44.063789 22477 catalog_manager.cc:1347] Generated new cluster ID: c2a78bfe681548e181acd89ebedfa6ae
I20260926 13:51:44.063942 22477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:51:44.095618 22477 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:51:44.097229 22477 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:51:44.104449 22477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626: Generated new TSK 0
I20260926 13:51:44.105482 22477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:51:44.872892 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:49913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:38161
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:51:45.029960 22508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:51:45.030565 22508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:51:45.030694 22508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:51:45.037272 22508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:51:45.037366 22508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:51:45.037487 22508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:51:45.045655 22508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.236.212:49913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:38161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:51:45.047075 22508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:51:45.048506 22508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:51:45.061939 22516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:51:45.061968 22518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:51:45.062412 22508 server_base.cc:1034] running on GCE node
W20260926 13:51:45.063316 22515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:51:45.063905 22508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:51:45.064702 22508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:51:45.065923 22508 hybrid_clock.cc:648] HybridClock initialized: now 1790430705065859 us; error 94 us; skew 500 ppm
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:51:45.069552 22508 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:51:45.070631 22508 webserver.cc:460] Webserver started at http://127.10.236.193:33113/ using document root <none> and password file <none>
I20260926 13:51:45.071337 22508 fs_manager.cc:359] Metadata directory not provided
I20260926 13:51:45.071408 22508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:51:45.071666 22508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:51:45.087558 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "63c96ed77b44458fa53b45a5626ab01c"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "39eef2ba92230f5f071ba1f1f8fba708"
server_key_iv: "1657da390050fbf2ad984d981ded0e07"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.088487 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "63c96ed77b44458fa53b45a5626ab01c"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "39eef2ba92230f5f071ba1f1f8fba708"
server_key_iv: "1657da390050fbf2ad984d981ded0e07"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.094388 22508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 13:51:45.131994 22527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.133980 22508 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260926 13:51:45.134207 22508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "63c96ed77b44458fa53b45a5626ab01c"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "39eef2ba92230f5f071ba1f1f8fba708"
server_key_iv: "1657da390050fbf2ad984d981ded0e07"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.134440 22508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:51:45.160327 22508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:51:45.164232 22508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:51:45.164427 22508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:51:45.165195 22508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:51:45.166126 22508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:51:45.166173 22508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.166263 22508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:51:45.166296 22508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:51:45.177552 22640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:35227 every 8 connection(s)
I20260926 13:51:45.177529 22508 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:35227
I20260926 13:51:45.178644 22508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:51:45.187862 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 22508
I20260926 13:51:45.188019 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:51:45.188279 11187 external_mini_cluster.cc:1428] Setting key 13c4d890b80925752d318bdbd2d18d22
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:51:45.199895 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.181768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:60847 (local address 127.10.236.254:38161)
0926 13:51:45.182015 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.182020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.182914 (+ 894us) server_negotiation.cc:407] Connection header received
0926 13:51:45.184000 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.184006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.184068 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.184170 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:51:45.185495 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.186052 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.187216 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.187449 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.187590 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.195178 (+ 7588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:51:45.195183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:51:45.195186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:51:45.195218 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:51:45.197222 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.197899 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.197903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.197905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.198006 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.198438 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.198442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.198444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.198699 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:51:45.198798 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.199568 (+ 770us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.199711 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":64,"threads_started":1}
I20260926 13:51:45.200950 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.182241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38161 (local address 127.10.236.193:60847)
0926 13:51:45.182727 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:45.182780 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:45.183771 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:45.184334 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:45.184348 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:45.184724 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:51:45.185351 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.185365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.186190 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.186193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.187085 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.187094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.192123 (+ 5029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.192129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.192375 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.193161 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:51:45.193184 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:51:45.194926 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:51:45.197363 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:51:45.197369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:51:45.197388 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:51:45.197764 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:51:45.198122 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:51:45.198126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:51:45.198128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:51:45.198322 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:51:45.198854 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:51:45.198862 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:51:45.199221 (+ 359us) client_negotiation.cc:769] Sending connection context
0926 13:51:45.199462 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 13:51:45.199881 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":154,"threads_started":1}
I20260926 13:51:45.202461 22641 heartbeater.cc:347] Connected to a master server at 127.10.236.254:38161
I20260926 13:51:45.202790 22641 heartbeater.cc:464] Registering TS with master...
I20260926 13:51:45.203370 22641 heartbeater.cc:511] Master 127.10.236.254:38161 requested a full tablet report, sending...
I20260926 13:51:45.205194 22427 ts_manager.cc:194] Registered new tserver with Master: 63c96ed77b44458fa53b45a5626ab01c (127.10.236.193:35227)
I20260926 13:51:45.206699 22427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:60847
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:51:45.233034 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:49913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:38161
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:51:45.362901 22647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:51:45.363207 22647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:51:45.363353 22647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:51:45.367952 22647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:51:45.368038 22647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:51:45.368139 22647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:51:45.374063 22647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.236.212:49913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:38161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:51:45.375221 22647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:51:45.376332 22647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:51:45.386442 22647 server_base.cc:1034] running on GCE node
W20260926 13:51:45.386364 22653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:51:45.386332 22654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:51:45.386667 22656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:51:45.387151 22647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:51:45.387902 22647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:51:45.389113 22647 hybrid_clock.cc:648] HybridClock initialized: now 1790430705389085 us; error 46 us; skew 500 ppm
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:51:45.392227 22647 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:51:45.393401 22647 webserver.cc:460] Webserver started at http://127.10.236.194:35429/ using document root <none> and password file <none>
I20260926 13:51:45.394136 22647 fs_manager.cc:359] Metadata directory not provided
I20260926 13:51:45.394240 22647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:51:45.394548 22647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:51:45.408733 22647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "176afc1eef7549bca41ce2d96afbdf90"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "e927c084395f43bca8365c26efa6f152"
server_key_iv: "cb37586fd2c0f77f6aae0ec39ada020b"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.409674 22647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "176afc1eef7549bca41ce2d96afbdf90"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "e927c084395f43bca8365c26efa6f152"
server_key_iv: "cb37586fd2c0f77f6aae0ec39ada020b"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.414785 22647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 13:51:45.444023 22663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.445559 22647 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260926 13:51:45.445713 22647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "176afc1eef7549bca41ce2d96afbdf90"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "e927c084395f43bca8365c26efa6f152"
server_key_iv: "cb37586fd2c0f77f6aae0ec39ada020b"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.445868 22647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:51:45.473351 22647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:51:45.480394 22647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:51:45.480580 22647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:51:45.481282 22647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:51:45.483989 22647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:51:45.484218 22647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.484314 22647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:51:45.484370 22647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.495361 22647 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:40443
I20260926 13:51:45.496608 22647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:51:45.501123 22776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:40443 every 8 connection(s)
I20260926 13:51:45.504876 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 22647
I20260926 13:51:45.505009 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:51:45.505323 11187 external_mini_cluster.cc:1428] Setting key c30deaae13756996821c760cc58cdb78
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:51:45.533046 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.505809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:44717 (local address 127.10.236.254:38161)
0926 13:51:45.505942 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.505947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.510012 (+ 4065us) server_negotiation.cc:407] Connection header received
0926 13:51:45.514160 (+ 4148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.514165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.514228 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.514284 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:51:45.516084 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.516953 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.523620 (+ 6667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.524007 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.524104 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.528687 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:51:45.528691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:51:45.528694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:51:45.528725 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:51:45.530477 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.531133 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.531137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.531139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.531230 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.531685 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.531688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.531690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.531918 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:51:45.532002 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.532725 (+ 723us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.532870 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:51:45.534796 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.503547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38161 (local address 127.10.236.194:44717)
0926 13:51:45.509840 (+ 6293us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:45.509905 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:45.511019 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:45.514777 (+ 3758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:45.514788 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:45.515155 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:51:45.515898 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.515919 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.517127 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.517131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.518513 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.518525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.524512 (+ 5987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.524517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.524769 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.526641 (+ 1872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:51:45.526664 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:51:45.528445 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:51:45.530637 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:51:45.530643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:51:45.530662 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:51:45.531013 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:51:45.531340 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:51:45.531343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:51:45.531345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:51:45.531572 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:51:45.532071 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:51:45.532078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:51:45.532419 (+ 341us) client_negotiation.cc:769] Sending connection context
0926 13:51:45.533332 (+ 913us) client_negotiation.cc:240] Negotiation successful
0926 13:51:45.533702 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6008,"thread_start_us":143,"threads_started":1}
I20260926 13:51:45.536373 22777 heartbeater.cc:347] Connected to a master server at 127.10.236.254:38161
I20260926 13:51:45.536653 22777 heartbeater.cc:464] Registering TS with master...
I20260926 13:51:45.537233 22777 heartbeater.cc:511] Master 127.10.236.254:38161 requested a full tablet report, sending...
I20260926 13:51:45.538375 22427 ts_manager.cc:194] Registered new tserver with Master: 176afc1eef7549bca41ce2d96afbdf90 (127.10.236.194:40443)
I20260926 13:51:45.539141 22427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:44717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:51:45.568955 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:49913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:38161
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
W20260926 13:51:45.743875 22782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:51:45.744385 22782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:51:45.744455 22782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:51:45.750255 22782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:51:45.750344 22782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:51:45.750429 22782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:51:45.758672 22782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.236.212:49913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:38161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:51:45.760254 22782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:51:45.761581 22782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:51:45.774646 22790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:51:45.775408 22788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:51:45.775799 22782 server_base.cc:1034] running on GCE node
W20260926 13:51:45.776084 22787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:51:45.776578 22782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:51:45.777386 22782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:51:45.785002 22782 hybrid_clock.cc:648] HybridClock initialized: now 1790430705784920 us; error 95 us; skew 500 ppm
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:51:45.788769 22782 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:51:45.789997 22782 webserver.cc:460] Webserver started at http://127.10.236.195:41773/ using document root <none> and password file <none>
I20260926 13:51:45.790730 22782 fs_manager.cc:359] Metadata directory not provided
I20260926 13:51:45.790861 22782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:51:45.791139 22782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:51:45.813653 22782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "8b17fd58dde947369727eca11987bb82"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "6a54aa445e026f167d73a076cba5cad3"
server_key_iv: "6c033f35fca61c433217e163773b88be"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.817979 22782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "8b17fd58dde947369727eca11987bb82"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "6a54aa445e026f167d73a076cba5cad3"
server_key_iv: "6c033f35fca61c433217e163773b88be"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.824198 22782 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260926 13:51:45.850729 22797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.852445 22782 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260926 13:51:45.852622 22782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "8b17fd58dde947369727eca11987bb82"
format_stamp: "Formatted at 2026-09-26 13:51:45 on dist-test-slave-r34j"
server_key: "6a54aa445e026f167d73a076cba5cad3"
server_key_iv: "6c033f35fca61c433217e163773b88be"
server_key_version: "kuduclusterkey@0"
I20260926 13:51:45.852792 22782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:51:45.870440 22782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:51:45.873407 22782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:51:45.873621 22782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:51:45.874277 22782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:51:45.875186 22782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:51:45.875255 22782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.875319 22782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:51:45.875375 22782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:51:45.887693 22782 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:35599
I20260926 13:51:45.889051 22782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:51:45.897742 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 22782
I20260926 13:51:45.897872 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:51:45.898176 11187 external_mini_cluster.cc:1428] Setting key 407e806e7428453c57598a5ce18fe0f9
I20260926 13:51:45.891965 22910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:35599 every 8 connection(s)
Sep 26 13:51:45 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:51:45.912371 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.895528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:37849 (local address 127.10.236.254:38161)
0926 13:51:45.895651 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.895656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.896416 (+ 760us) server_negotiation.cc:407] Connection header received
0926 13:51:45.897684 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.897688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.897745 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.897798 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:51:45.901256 (+ 3458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.902000 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.903594 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.903901 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.903985 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.906982 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:51:45.906987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:51:45.906989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:51:45.907020 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:51:45.908782 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.909417 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.909420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.909422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.909512 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.909896 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.909899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.909901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.910124 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:51:45.910200 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.911437 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.912213 (+ 776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:51:45.913982 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.894369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38161 (local address 127.10.236.195:37849)
0926 13:51:45.896250 (+ 1881us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:45.896305 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:45.897415 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:45.898220 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:45.898228 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:45.899263 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:51:45.901091 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.901113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.902156 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.902159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.903439 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.903450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.904183 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.904186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.904369 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.905083 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:51:45.905105 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:51:45.906770 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:51:45.908927 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:51:45.908945 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:51:45.908964 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:51:45.909303 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:51:45.909613 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:51:45.909616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:51:45.909618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:51:45.909798 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:51:45.910263 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:51:45.910269 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:51:45.910585 (+ 316us) client_negotiation.cc:769] Sending connection context
0926 13:51:45.912577 (+ 1992us) client_negotiation.cc:240] Negotiation successful
0926 13:51:45.912928 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1668,"thread_start_us":162,"threads_started":1}
I20260926 13:51:45.915167 22911 heartbeater.cc:347] Connected to a master server at 127.10.236.254:38161
I20260926 13:51:45.915405 22911 heartbeater.cc:464] Registering TS with master...
I20260926 13:51:45.915992 22911 heartbeater.cc:511] Master 127.10.236.254:38161 requested a full tablet report, sending...
I20260926 13:51:45.917037 22427 ts_manager.cc:194] Registered new tserver with Master: 8b17fd58dde947369727eca11987bb82 (127.10.236.195:35599)
I20260926 13:51:45.917732 22427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:37849
I20260926 13:51:45.923830 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:51:45.935891 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.926286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.10.236.254:38161)
0926 13:51:45.926396 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.926401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.926968 (+ 567us) server_negotiation.cc:407] Connection header received
0926 13:51:45.927141 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.927144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.927189 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.927236 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:51:45.928227 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.928996 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.930197 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.930518 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.930592 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.932405 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:51:45.932409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:51:45.932412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:51:45.932437 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:51:45.934083 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.934561 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.934564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.934566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.934646 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:51:45.934926 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:51:45.934929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:51:45.934931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:51:45.935141 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:51:45.935241 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.935618 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.935734 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:51:45.939992 22427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:51:45.944690 22427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:51:45.975555 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.960511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35599 (local address 127.0.0.1:51530)
0926 13:51:45.963949 (+ 3438us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:45.963962 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:45.964050 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:45.965083 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:45.965089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:45.965138 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:51:45.965300 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.965308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.968736 (+ 3428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.968740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.972444 (+ 3704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.972455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.974912 (+ 2457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.974916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.975072 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.975116 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 13:51:45.975224 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 13:51:45.975360 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3382,"thread_start_us":118,"threads_started":1}
I20260926 13:51:45.979032 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.958102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:40443 (local address 127.0.0.1:54648)
0926 13:51:45.963288 (+ 5186us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:45.963351 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:45.963512 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:45.967048 (+ 3536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:45.967051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:45.967079 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:51:45.967198 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.967204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.975962 (+ 8758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.975965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.977222 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.977232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.978616 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.978620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.978721 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.978744 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:51:45.978813 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:51:45.978880 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4986,"thread_start_us":109,"threads_started":1}
I20260926 13:51:45.980719 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.961909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35227 (local address 127.0.0.1:37318)
0926 13:51:45.964216 (+ 2307us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:45.964228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:45.964286 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:45.976591 (+ 12305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:45.976594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:45.976615 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:51:45.976731 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.976736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.978129 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.978133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.979209 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:45.979218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.980308 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.980312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:45.980416 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.980439 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:51:45.980490 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 13:51:45.980578 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2259,"thread_start_us":104,"threads_started":1}
I20260926 13:51:45.981117 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.961106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51530 (local address 127.10.236.195:35599)
0926 13:51:45.963816 (+ 2710us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.963828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.964378 (+ 550us) server_negotiation.cc:407] Connection header received
0926 13:51:45.964469 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.964475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.964717 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.964882 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:51:45.967378 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.968602 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.972699 (+ 4097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.973728 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.973874 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.974041 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.980805 (+ 6764us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.980943 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2595,"thread_start_us":160,"threads_started":1}
I20260926 13:51:45.981774 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.962093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.10.236.193:35227)
0926 13:51:45.975781 (+ 13688us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.975788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.975815 (+ 27us) server_negotiation.cc:407] Connection header received
0926 13:51:45.975900 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.975908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.976151 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.976309 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:51:45.976874 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.977980 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.979362 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.980184 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.981379 (+ 1195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.981511 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.981558 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.981644 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13551,"thread_start_us":132,"threads_started":1}
I20260926 13:51:45.984477 22838 tablet_service.cc:1467] Processing CreateTablet for tablet 7a832977a19542d39fb3fe11feee2187 (DEFAULT_TABLE table=test-table [id=ecb8331100ca471e8716c179dfe62906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:51:45.984470 22575 tablet_service.cc:1467] Processing CreateTablet for tablet 7a832977a19542d39fb3fe11feee2187 (DEFAULT_TABLE table=test-table [id=ecb8331100ca471e8716c179dfe62906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:51:45.985666 22575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a832977a19542d39fb3fe11feee2187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:51:45.985666 22838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a832977a19542d39fb3fe11feee2187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:51:45.997473 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:45.959095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.10.236.194:40443)
0926 13:51:45.966411 (+ 7316us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:45.966428 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:45.966453 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:51:45.966527 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:45.966532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:45.966753 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:45.966903 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:51:45.970249 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.971420 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.977401 (+ 5981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:45.978286 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:45.978404 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:45.978544 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:45.997149 (+ 18605us) server_negotiation.cc:299] Negotiation successful
0926 13:51:45.997296 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7196,"thread_start_us":72,"threads_started":1}
I20260926 13:51:45.999895 22931 tablet_bootstrap.cc:492] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: Bootstrap starting.
I20260926 13:51:46.000478 22692 tablet_service.cc:1467] Processing CreateTablet for tablet 7a832977a19542d39fb3fe11feee2187 (DEFAULT_TABLE table=test-table [id=ecb8331100ca471e8716c179dfe62906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:51:46.001488 22930 tablet_bootstrap.cc:492] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Bootstrap starting.
I20260926 13:51:46.001688 22692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a832977a19542d39fb3fe11feee2187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:51:46.003196 22931 tablet_bootstrap.cc:654] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: Neither blocks nor log segments found. Creating new log.
I20260926 13:51:46.005934 22930 tablet_bootstrap.cc:654] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Neither blocks nor log segments found. Creating new log.
I20260926 13:51:46.008395 22931 log.cc:824] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:51:46.011749 22932 tablet_bootstrap.cc:492] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Bootstrap starting.
I20260926 13:51:46.013268 22930 log.cc:824] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Log is configured to *not* fsync() on all Append() calls
I20260926 13:51:46.015319 22932 tablet_bootstrap.cc:654] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Neither blocks nor log segments found. Creating new log.
I20260926 13:51:46.016249 22932 log.cc:824] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Log is configured to *not* fsync() on all Append() calls
I20260926 13:51:46.018543 22932 tablet_bootstrap.cc:492] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: No bootstrap required, opened a new log
I20260926 13:51:46.018793 22932 ts_tablet_manager.cc:1397] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 13:51:46.023057 22932 raft_consensus.cc:348] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.023348 22932 raft_consensus.cc:374] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:51:46.023396 22932 raft_consensus.cc:729] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176afc1eef7549bca41ce2d96afbdf90, State: Initialized, Role: FOLLOWER
I20260926 13:51:46.025079 22932 consensus_queue.cc:260] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.026328 22932 ts_tablet_manager.cc:1428] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 13:51:46.026719 22777 heartbeater.cc:503] Master 127.10.236.254:38161 was elected leader, sending a full tablet report...
I20260926 13:51:46.031706 22931 tablet_bootstrap.cc:492] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: No bootstrap required, opened a new log
I20260926 13:51:46.032048 22931 ts_tablet_manager.cc:1397] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: Time spent bootstrapping tablet: real 0.032s user 0.006s sys 0.000s
I20260926 13:51:46.037231 22930 tablet_bootstrap.cc:492] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: No bootstrap required, opened a new log
I20260926 13:51:46.037537 22930 ts_tablet_manager.cc:1397] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.000s
I20260926 13:51:46.042768 22930 raft_consensus.cc:348] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.046146 22931 raft_consensus.cc:348] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.046561 22931 raft_consensus.cc:374] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:51:46.046666 22931 raft_consensus.cc:729] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c96ed77b44458fa53b45a5626ab01c, State: Initialized, Role: FOLLOWER
I20260926 13:51:46.047295 22931 consensus_queue.cc:260] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.047559 22930 raft_consensus.cc:374] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:51:46.047673 22930 raft_consensus.cc:729] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b17fd58dde947369727eca11987bb82, State: Initialized, Role: FOLLOWER
I20260926 13:51:46.049103 22931 ts_tablet_manager.cc:1428] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260926 13:51:46.049479 22930 consensus_queue.cc:260] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.050529 22930 ts_tablet_manager.cc:1428] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260926 13:51:46.050958 22641 heartbeater.cc:503] Master 127.10.236.254:38161 was elected leader, sending a full tablet report...
I20260926 13:51:46.054056 22911 heartbeater.cc:503] Master 127.10.236.254:38161 was elected leader, sending a full tablet report...
Sep 26 13:51:46 dist-test-slave-r34j krb5kdc[21951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430705, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
W20260926 13:51:46.146319 22912 tablet.cc:2369] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:51:46.151170 22936 raft_consensus.cc:482] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:51:46.151396 22936 raft_consensus.cc:504] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.152813 22936 leader_election.cc:288] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b17fd58dde947369727eca11987bb82 (127.10.236.195:35599), 63c96ed77b44458fa53b45a5626ab01c (127.10.236.193:35227)
I20260926 13:51:46.159729 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:46.153566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:56453 (local address 127.10.236.195:35599)
0926 13:51:46.153694 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:46.153699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:46.154992 (+ 1293us) server_negotiation.cc:407] Connection header received
0926 13:51:46.155184 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:46.155188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:46.155232 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:46.155278 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:51:46.155729 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.156489 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.158193 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.158905 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.159013 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:46.159068 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:46.159437 (+ 369us) server_negotiation.cc:299] Negotiation successful
0926 13:51:46.159589 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:51:46.160207 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:46.153174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35599 (local address 127.10.236.194:56453)
0926 13:51:46.154907 (+ 1733us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:46.154924 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:46.155070 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:46.155432 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:46.155435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:46.155452 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:51:46.155589 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:46.155598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.156713 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.156716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:46.158042 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:46.158055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.159180 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.159183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:46.159278 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:46.159302 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:51:46.159961 (+ 659us) client_negotiation.cc:240] Negotiation successful
0926 13:51:46.160045 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1663,"thread_start_us":101,"threads_started":1}
I20260926 13:51:46.160902 22847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a832977a19542d39fb3fe11feee2187" candidate_uuid: "176afc1eef7549bca41ce2d96afbdf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b17fd58dde947369727eca11987bb82" is_pre_election: true
I20260926 13:51:46.161283 22847 raft_consensus.cc:2454] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176afc1eef7549bca41ce2d96afbdf90 in term 0.
I20260926 13:51:46.161908 22666 leader_election.cc:302] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176afc1eef7549bca41ce2d96afbdf90, 8b17fd58dde947369727eca11987bb82; no voters:
I20260926 13:51:46.162236 22936 raft_consensus.cc:2781] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:51:46.162323 22936 raft_consensus.cc:482] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:51:46.162367 22936 raft_consensus.cc:3037] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:51:46.163833 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:46.154283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:54603 (local address 127.10.236.193:35227)
0926 13:51:46.154403 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:46.154423 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:46.156804 (+ 2381us) server_negotiation.cc:407] Connection header received
0926 13:51:46.158865 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:46.158869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:46.158927 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:46.158977 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:51:46.159395 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.160196 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.162441 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.163079 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.163180 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:46.163230 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:46.163631 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 13:51:46.163714 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:51:46.163800 22936 raft_consensus.cc:504] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.164192 22936 leader_election.cc:288] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [CANDIDATE]: Term 1 election: Requested vote from peers 8b17fd58dde947369727eca11987bb82 (127.10.236.195:35599), 63c96ed77b44458fa53b45a5626ab01c (127.10.236.193:35227)
I20260926 13:51:46.164654 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:46.153920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35227 (local address 127.10.236.194:54603)
0926 13:51:46.156704 (+ 2784us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:46.156722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:51:46.158712 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:51:46.159117 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:51:46.159119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:51:46.159136 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:51:46.159262 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:46.159269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.160354 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.160357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:46.161575 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:51:46.161598 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.163335 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.163338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:51:46.163443 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:46.163465 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:51:46.164017 (+ 552us) client_negotiation.cc:240] Negotiation successful
0926 13:51:46.164511 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2715,"thread_start_us":96,"threads_started":1}
I20260926 13:51:46.165491 22595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a832977a19542d39fb3fe11feee2187" candidate_uuid: "176afc1eef7549bca41ce2d96afbdf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c96ed77b44458fa53b45a5626ab01c" is_pre_election: true
I20260926 13:51:46.165797 22595 raft_consensus.cc:2454] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176afc1eef7549bca41ce2d96afbdf90 in term 0.
I20260926 13:51:46.166179 22594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a832977a19542d39fb3fe11feee2187" candidate_uuid: "176afc1eef7549bca41ce2d96afbdf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63c96ed77b44458fa53b45a5626ab01c"
I20260926 13:51:46.166289 22594 raft_consensus.cc:3037] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:51:46.167665 22594 raft_consensus.cc:2454] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176afc1eef7549bca41ce2d96afbdf90 in term 1.
I20260926 13:51:46.168488 22667 leader_election.cc:302] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176afc1eef7549bca41ce2d96afbdf90, 63c96ed77b44458fa53b45a5626ab01c; no voters:
I20260926 13:51:46.168540 22847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a832977a19542d39fb3fe11feee2187" candidate_uuid: "176afc1eef7549bca41ce2d96afbdf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b17fd58dde947369727eca11987bb82"
I20260926 13:51:46.168670 22847 raft_consensus.cc:3037] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:51:46.169991 22847 raft_consensus.cc:2454] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176afc1eef7549bca41ce2d96afbdf90 in term 1.
I20260926 13:51:46.170224 22936 raft_consensus.cc:2781] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:51:46.170449 22936 raft_consensus.cc:686] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 1 LEADER]: Becoming Leader. State: Replica: 176afc1eef7549bca41ce2d96afbdf90, State: Running, Role: LEADER
I20260926 13:51:46.170847 22936 consensus_queue.cc:237] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } }
I20260926 13:51:46.174949 22426 catalog_manager.cc:5495] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to 176afc1eef7549bca41ce2d96afbdf90 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "176afc1eef7549bca41ce2d96afbdf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176afc1eef7549bca41ce2d96afbdf90" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 40443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 } health_report { overall_health: UNKNOWN } } }
W20260926 13:51:46.183673 22642 tablet.cc:2369] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:51:46.212388 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:46.207384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.10.236.194:40443)
0926 13:51:46.207541 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:51:46.207545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:51:46.207561 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:51:46.207655 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:51:46.207658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:51:46.207735 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:51:46.207862 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:51:46.208184 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.208946 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.210200 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:51:46.210522 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:51:46.210638 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:51:46.211088 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:51:46.211720 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:51:46.211842 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 13:51:46.212191 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 13:51:46.212252 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:51:46.221414 22594 raft_consensus.cc:1261] T 7a832977a19542d39fb3fe11feee2187 P 63c96ed77b44458fa53b45a5626ab01c [term 1 FOLLOWER]: Refusing update from remote peer 176afc1eef7549bca41ce2d96afbdf90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:51:46.222487 22936 consensus_queue.cc:1035] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63c96ed77b44458fa53b45a5626ab01c" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:51:46.223011 22847 raft_consensus.cc:1261] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 1 FOLLOWER]: Refusing update from remote peer 176afc1eef7549bca41ce2d96afbdf90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:51:46.223694 22943 consensus_queue.cc:1035] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b17fd58dde947369727eca11987bb82" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:51:46.245879 22948 mvcc.cc:205] Tried to move back new op lower bound from 7333604172672077824 to 7333604172480524288. Current Snapshot: MvccSnapshot[applied={T|T < 7333604172672077824 or (T in {7333604172672077824})}]
I20260926 13:51:46.259179 22426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:51:46.259374 22426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:51:46.262434 22426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5187d92aec5b4ce3bca0f3603fc0b626: Sending DeleteTablet for 3 replicas of tablet 7a832977a19542d39fb3fe11feee2187
I20260926 13:51:46.263387 22838 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a832977a19542d39fb3fe11feee2187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:51:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51530
I20260926 13:51:46.263371 22575 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a832977a19542d39fb3fe11feee2187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:51:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:37318
I20260926 13:51:46.264417 22692 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a832977a19542d39fb3fe11feee2187 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:51:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:54648
I20260926 13:51:46.265863 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 22508
I20260926 13:51:46.270563 22958 tablet_replica.cc:330] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: stopping tablet replica
I20260926 13:51:46.270915 22958 raft_consensus.cc:2229] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:51:46.271359 22958 raft_consensus.cc:2258] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:51:46.272993 22958 ts_tablet_manager.cc:1905] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:51:46.276405 22958 ts_tablet_manager.cc:1918] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:51:46.276516 22958 log.cc:1196] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/7a832977a19542d39fb3fe11feee2187
I20260926 13:51:46.276835 22958 ts_tablet_manager.cc:1939] T 7a832977a19542d39fb3fe11feee2187 P 176afc1eef7549bca41ce2d96afbdf90: Deleting consensus metadata
I20260926 13:51:46.277149 22956 tablet_replica.cc:330] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: stopping tablet replica
I20260926 13:51:46.278144 22412 catalog_manager.cc:4841] TS 176afc1eef7549bca41ce2d96afbdf90 (127.10.236.194:40443): tablet 7a832977a19542d39fb3fe11feee2187 (table test-table [id=ecb8331100ca471e8716c179dfe62906]) successfully deleted
I20260926 13:51:46.278635 22956 raft_consensus.cc:2229] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:51:46.278978 22956 raft_consensus.cc:2258] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:51:46.280750 22956 ts_tablet_manager.cc:1905] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:51:46.283742 22956 ts_tablet_manager.cc:1918] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:51:46.283900 22956 log.cc:1196] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/7a832977a19542d39fb3fe11feee2187
I20260926 13:51:46.284384 22956 ts_tablet_manager.cc:1939] T 7a832977a19542d39fb3fe11feee2187 P 8b17fd58dde947369727eca11987bb82: Deleting consensus metadata
I20260926 13:51:46.285955 22413 catalog_manager.cc:4841] TS 8b17fd58dde947369727eca11987bb82 (127.10.236.195:35599): tablet 7a832977a19542d39fb3fe11feee2187 (table test-table [id=ecb8331100ca471e8716c179dfe62906]) successfully deleted
W20260926 13:51:46.293100 22414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:35227 (error 108)
I20260926 13:51:46.299978 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 22647
I20260926 13:51:46.311349 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:51:46.311009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35227 (local address 127.0.0.1:37330)
0926 13:51:46.311143 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:51:46.311232 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:35227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 13:51:46.311652 22414 catalog_manager.cc:4584] TS 63c96ed77b44458fa53b45a5626ab01c (127.10.236.193:35227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a832977a19542d39fb3fe11feee2187: Network error: Client connection negotiation failed: client connection to 127.10.236.193:35227: connect: Connection refused (error 111)
I20260926 13:51:46.311852 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 22782
I20260926 13:51:46.319029 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 22395
I20260926 13:51:46.325008 11187 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 13:51:46.670400 11187 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 13:51:46.670518 11187 mini_ranger.cc:66] Stopping Ranger...
I20260926 13:51:47.016628 11187 mini_ranger.cc:68] Stopped Ranger
2026-09-26 13:51:47.016 UTC [21981] LOG: received smart shutdown request
2026-09-26 13:51:47.021 UTC [21981] LOG: background worker "logical replication launcher" (PID 21990) exited with exit code 1
2026-09-26 13:51:47.021 UTC [21985] LOG: shutting down
2026-09-26 13:51:47.226 UTC [21981] LOG: database system is shut down
2026-09-26T13:51:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48516 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:51:47 dist-test-slave-r34j krb5kdc[22968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:51:47 dist-test-slave-r34j krb5kdc[22968](info): set up 2 sockets
Sep 26 13:51:47 dist-test-slave-r34j krb5kdc[22968](info): commencing operation
krb5kdc: starting...
W20260926 13:51:48.416090 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:51:48 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:51:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/rangeradmin_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/rangeradmin_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/rangerlookup_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/rangerlookup_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
I20260926 13:51:48.619645 11187 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskMS3Khi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/postgres -l logfile start
2026-09-26 13:51:51.657 UTC [22998] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 13:51:51.657 UTC [22998] LOG: listening on IPv4 address "127.10.236.212", port 49905
2026-09-26 13:51:51.662 UTC [22998] LOG: listening on Unix socket "/tmp/.s.PGSQL.49905"
2026-09-26 13:51:51.701 UTC [23001] LOG: database system was shut down at 2026-09-26 13:51:49 UTC
2026-09-26 13:51:51.705 UTC [22998] LOG: database system is ready to accept connections
I20260926 13:51:52.699389 11187 mini_postgres.cc:96] Postgres bound to 49905
2026-09-26 13:51:52.705 UTC [23009] FATAL: database "slave" does not exist
127.10.236.212:49905 - accepting connections
I20260926 13:51:52.706249 11187 mini_ranger.cc:161] Starting Ranger...
I20260926 13:51:52.721771 11187 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 13:51:53.342792 11187 mini_ranger.cc:87] Created ranger Postgres database
I20260926 13:51:53.342862 11187 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-admin
2026-09-26 13:51:53,624 [I] DB FLAVOR :POSTGRES
2026-09-26 13:51:53,625 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:51:53,625 [I] Checking connection..
2026-09-26 13:51:53,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 13:51:53,902 [I] Checking connection passed.
2026-09-26 13:51:53,902 [I] --------- Verifying version history table ---------
2026-09-26 13:51:53,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:51:54,145 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:51:54,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:51:54,390 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:51:54,390 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 13:51:54,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 13:51:54.637 UTC [23073] WARNING: there is no transaction in progress
2026-09-26 13:51:54,644 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 13:51:54,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:51:54,899 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 13:51:54,899 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:51:54,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 13:51:55,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 13:51:55,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-r34j.c.gcp-upstream.internal','N') ;"
2026-09-26 13:51:55,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:51:55,889 [I] Table x_portal_user does not exist in database ranger
2026-09-26 13:51:55,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:51:56,143 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 13:51:56,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:51:56,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:51:56,666 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 13:51:56,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 13:51:57.758 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.777 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.797 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.818 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.838 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.860 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.923 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.932 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.949 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.960 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.972 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:57.990 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:58.001 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:58.008 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:58.017 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:58.026 UTC [23181] WARNING: there is no transaction in progress
2026-09-26 13:51:58,782 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 13:51:58,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 13:51:59,014 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 13:51:59,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:51:59,281 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 13:51:59,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:51:59,550 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 13:51:59,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:51:59,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:52:00,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-r34j.c.gcp-upstream.internal';"
2026-09-26 13:52:00,346 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 13:52:00,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 13:52:00,606 [I] DB_PATCHES have already been applied
I20260926 13:52:00.612833 11187 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-admin:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskMS3Khi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 13:52:00.612903 11187 mini_ranger.cc:193] Using host: 127.10.236.212
I20260926 13:52:00.616709 11187 mini_ranger.cc:227] Ranger admin URL: http://127.10.236.212:49765
Sep 26, 2026 1:52:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:52:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:52:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 1:52:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 1:52:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 1:52:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 1:52:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 13:52:01.744899 11187 mini_ranger.cc:160] Time spent starting Ranger: real 9.039s user 0.003s sys 0.007s
Sep 26, 2026 1:52:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 1:52:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 1:52:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 1:52:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 1:52:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 13:52:24.705396 11187 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.236.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.236.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/rangerkms_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.236.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/rangerkms_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.236.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.236.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.236.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/HTTP_127.10.236.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 13:52:24 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430744, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 13:52:24.808777 11187 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 13:52:24.819072 11187 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 13:52:25.904655 11187 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 13:52:25.904747 11187 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms
I20260926 13:52:25.904769 11187 mini_ranger_kms.cc:227] Using postgres at 127.10.236.212:49905
2026-09-26 13:52:26,086 [I] DB FLAVOR :POSTGRES
2026-09-26 13:52:26,086 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:52:26,086 [I] Checking connection
2026-09-26 13:52:26,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 13:52:26,353 [I] connection success
2026-09-26 13:52:26,354 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 13:52:26,354 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 13:52:26,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 13:52:26,614 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 13:52:26,614 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:52:26,614 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 13:52:26,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.236.212:49905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 13:52:26,873 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 13:52:27.380213 11187 mini_ranger_kms.cc:310] Created kms service
I20260926 13:52:27.598096 11187 mini_ranger_kms.cc:326] Created kudu user
I20260926 13:52:27.647603 11187 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 13:52:27 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430744, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:28.056630 11187 mini_ranger_kms.cc:384] Added ranger policy
I20260926 13:52:28.056901 11187 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms:/tmp/dist-test-taskMS3Khi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskMS3Khi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 13:52:28.056932 11187 mini_ranger_kms.cc:240] Using host: 127.10.236.212
I20260926 13:52:28.061138 11187 mini_ranger_kms.cc:276] Ranger KMS PID: 23362
I20260926 13:52:28.061239 11187 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.236.212:51265
Sep 26, 2026 1:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 1:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 1:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 1:52:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 13:52:30.345731 11187 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.537s user 0.012s sys 0.154s
I20260926 13:52:30.345870 11187 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 13:52:30.345968 11187 mini_ranger_kms.cc:401] 127.10.236.212:51265/kms/v1/keys
Sep 26 13:52:30 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790430750, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.236.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 13:52:30 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430750, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.236.212@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
Sep 26 13:52:31 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430750, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.236.212@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
Sep 26 13:52:33 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:52:33.536947 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37367
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:37367
--ranger_config_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:33.652086 23412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:33.652444 23412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:33.652489 23412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:33.656656 23412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:33.656728 23412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:33.656749 23412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:33.656765 23412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:33.656800 23412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:33.662595 23412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.236.212:51265/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:37367
--ranger_config_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:33.663765 23412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:33.664868 23412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:33.673786 23417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:33.673815 23420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:33.674146 23412 server_base.cc:1034] running on GCE node
W20260926 13:52:33.673789 23418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:33.674921 23412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:33.676223 23412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:33.677433 23412 hybrid_clock.cc:648] HybridClock initialized: now 1790430753677382 us; error 66 us; skew 500 ppm
Sep 26 13:52:33 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430753, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:33.689987 23412 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:33.691078 23412 webserver.cc:460] Webserver started at http://127.10.236.254:45271/ using document root <none> and password file <none>
I20260926 13:52:33.691733 23412 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:33.691814 23412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:33.692044 23412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:52:33 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430753, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:33.754292 23412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "cd0f1bf5d2ba446ab78b2d4594851662"
format_stamp: "Formatted at 2026-09-26 13:52:33 on dist-test-slave-r34j"
server_key: "fd1cdb77b6b984a6526b6005934b4297"
server_key_iv: "16268e44b5eb3899d3bcd7b6aac28cfa"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:33.754940 23412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "cd0f1bf5d2ba446ab78b2d4594851662"
format_stamp: "Formatted at 2026-09-26 13:52:33 on dist-test-slave-r34j"
server_key: "fd1cdb77b6b984a6526b6005934b4297"
server_key_iv: "16268e44b5eb3899d3bcd7b6aac28cfa"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:33.759315 23412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260926 13:52:33.780236 23427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:33.781972 23412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:33.782145 23412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "cd0f1bf5d2ba446ab78b2d4594851662"
format_stamp: "Formatted at 2026-09-26 13:52:33 on dist-test-slave-r34j"
server_key: "fd1cdb77b6b984a6526b6005934b4297"
server_key_iv: "16268e44b5eb3899d3bcd7b6aac28cfa"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:33.782286 23412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:33.824987 23412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:33.828219 23412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:33.828418 23412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:33.836705 23412 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:37367
I20260926 13:52:33.836709 23479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:37367 every 8 connection(s)
I20260926 13:52:33.837752 23412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:33.841207 23480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:33.845935 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 23412
I20260926 13:52:33.846074 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:52:33.846349 11187 external_mini_cluster.cc:1428] Setting key d736f15d9c93ae8c78414a2fb96168bd
I20260926 13:52:33.848757 23480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Bootstrap starting.
I20260926 13:52:33.851838 23480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:33.852826 23480 log.cc:824] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:33.854911 23480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: No bootstrap required, opened a new log
Sep 26 13:52:33 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:33.859051 23480 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:33.859373 23480 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:33.859472 23480 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0f1bf5d2ba446ab78b2d4594851662, State: Initialized, Role: FOLLOWER
I20260926 13:52:33.860028 23480 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:33.860204 23480 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:52:33.860297 23480 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:52:33.860419 23480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:33.861647 23480 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:33.862000 23480 leader_election.cc:302] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0f1bf5d2ba446ab78b2d4594851662; no voters:
I20260926 13:52:33.862350 23480 leader_election.cc:288] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:52:33.862423 23485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:33.862463 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:33.847882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50436 (local address 127.10.236.254:37367)
0926 13:52:33.848295 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:33.848312 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:33.848347 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:52:33.849037 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:33.849056 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:33.849497 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:33.849878 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:33.850693 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:33.851781 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:33.852674 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:33.853213 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:33.853394 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:33.856708 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:33.856716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:33.856734 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:33.856768 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:33.858903 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:33.859528 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:33.859535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:33.859542 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:33.859694 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:33.860038 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:33.860042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:33.860044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:33.860568 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:33.860798 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:33.861073 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 13:52:33.861439 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260926 13:52:33.862686 23485 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 LEADER]: Becoming Leader. State: Replica: cd0f1bf5d2ba446ab78b2d4594851662, State: Running, Role: LEADER
I20260926 13:52:33.863093 23485 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:33.863698 23480 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:33.864789 23486 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0f1bf5d2ba446ab78b2d4594851662. Latest consensus state: current_term: 1 leader_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } } }
I20260926 13:52:33.864897 23486 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:33.865357 23487 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } } }
I20260926 13:52:33.865489 23487 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:33.865825 23480 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 13:52:33.870819 23494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:33.874980 23494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:33.882128 23494 catalog_manager.cc:1347] Generated new cluster ID: b553352342a54330a51f2f155d9cd8be
I20260926 13:52:33.882434 23494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:52:33.894928 23494 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:52:33.896385 23494 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:52:33.909188 23494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Generated new TSK 0
I20260926 13:52:33.910178 23494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:52:34.650838 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37367
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:34.805881 23525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:34.806272 23525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:34.806347 23525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:34.810894 23525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:34.810957 23525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:34.811024 23525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:52:34.816689 23525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.236.212:51265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:34.817927 23525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:34.819005 23525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:34.829370 23530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:34.829874 23531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:34.830193 23525 server_base.cc:1034] running on GCE node
W20260926 13:52:34.830325 23533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:34.830840 23525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:34.831589 23525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:34.832803 23525 hybrid_clock.cc:648] HybridClock initialized: now 1790430754832767 us; error 48 us; skew 500 ppm
Sep 26 13:52:34 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430754, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:34.837462 23525 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:52:34.838721 23525 webserver.cc:460] Webserver started at http://127.10.236.193:37211/ using document root <none> and password file <none>
I20260926 13:52:34.839561 23525 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:34.839705 23525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:34.839998 23525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:52:34 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430754, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:34.860829 23525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "2ebb4e47adbc4b46a2ca564235d3f038"
format_stamp: "Formatted at 2026-09-26 13:52:34 on dist-test-slave-r34j"
server_key: "e77871580df44c9650e7af56ef48f642"
server_key_iv: "7c43edd4755a2c59a4dfd7586e6b2802"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:34.861693 23525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "2ebb4e47adbc4b46a2ca564235d3f038"
format_stamp: "Formatted at 2026-09-26 13:52:34 on dist-test-slave-r34j"
server_key: "e77871580df44c9650e7af56ef48f642"
server_key_iv: "7c43edd4755a2c59a4dfd7586e6b2802"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:34.867238 23525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 13:52:34.891880 23542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:34.893592 23525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 13:52:34.893764 23525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2ebb4e47adbc4b46a2ca564235d3f038"
format_stamp: "Formatted at 2026-09-26 13:52:34 on dist-test-slave-r34j"
server_key: "e77871580df44c9650e7af56ef48f642"
server_key_iv: "7c43edd4755a2c59a4dfd7586e6b2802"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:34.893923 23525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:34.912374 23525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:34.916333 23525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:34.916620 23525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:34.917594 23525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:34.918848 23525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:34.918962 23525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:34.919102 23525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:34.919181 23525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:34.931087 23525 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:34803
I20260926 13:52:34.932389 23525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:52:34.933225 23655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:34803 every 8 connection(s)
I20260926 13:52:34.941306 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 23525
I20260926 13:52:34.941465 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:52:34.941766 11187 external_mini_cluster.cc:1428] Setting key cd525b7227de66bc7acd857cc562dc68
Sep 26 13:52:34 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430754, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:34.953615 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:34.936418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37367 (local address 127.10.236.193:59211)
0926 13:52:34.936932 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:34.936986 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:34.938042 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:34.938854 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:34.938863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:34.939210 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:34.939909 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:34.939929 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:34.941191 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:34.941194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:34.943805 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:34.943819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:34.944643 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:34.944648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:34.944853 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:34.945584 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:34.945607 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:34.947384 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:34.949876 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:34.949883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:34.949903 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:34.950284 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:34.950644 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:34.950647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:34.950649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:34.950839 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:34.951369 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:34.951376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:34.951771 (+ 395us) client_negotiation.cc:769] Sending connection context
0926 13:52:34.951987 (+ 216us) client_negotiation.cc:240] Negotiation successful
0926 13:52:34.952397 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":153,"threads_started":1}
I20260926 13:52:34.954406 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:34.935993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:59211 (local address 127.10.236.254:37367)
0926 13:52:34.936255 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:34.936259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:34.937091 (+ 832us) server_negotiation.cc:407] Connection header received
0926 13:52:34.938310 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:34.938314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:34.938373 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:34.938434 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:34.940089 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:34.941008 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:34.943961 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:34.944342 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:34.944457 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:34.947645 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:34.947649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:34.947652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:34.947680 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:34.949727 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:34.950433 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:34.950436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:34.950438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:34.950534 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:34.950952 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:34.950955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:34.950957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:34.951195 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:34.951312 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:34.954086 (+ 2774us) server_negotiation.cc:299] Negotiation successful
0926 13:52:34.954241 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260926 13:52:34.956409 23656 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37367
I20260926 13:52:34.956650 23656 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:34.957252 23656 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:34.959182 23444 ts_manager.cc:194] Registered new tserver with Master: 2ebb4e47adbc4b46a2ca564235d3f038 (127.10.236.193:34803)
I20260926 13:52:34.960744 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:59211
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:52:34.988837 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37367
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 13:52:35.143235 23664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:35.143585 23664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:35.143674 23664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:35.148856 23664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:35.148952 23664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:35.149061 23664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:52:35.155411 23664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.236.212:51265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:35.156617 23664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:35.157681 23664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:35.167970 23671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:35.167970 23673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:35.168314 23670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:35.168713 23664 server_base.cc:1034] running on GCE node
I20260926 13:52:35.169123 23664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:35.169862 23664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:35.171036 23664 hybrid_clock.cc:648] HybridClock initialized: now 1790430755171005 us; error 53 us; skew 500 ppm
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:35.174057 23664 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:52:35.175037 23664 webserver.cc:460] Webserver started at http://127.10.236.194:37087/ using document root <none> and password file <none>
I20260926 13:52:35.175824 23664 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:35.175920 23664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:35.176157 23664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:35.196547 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "ca116f6095244f5ba146e7789a220a88"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "26f6e5073e308ace437ff19ed04d42ee"
server_key_iv: "c18d4da54d439f715ff235e66dad741c"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.197345 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "ca116f6095244f5ba146e7789a220a88"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "26f6e5073e308ace437ff19ed04d42ee"
server_key_iv: "c18d4da54d439f715ff235e66dad741c"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.205986 23664 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260926 13:52:35.224959 23680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.227411 23664 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260926 13:52:35.227727 23664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "ca116f6095244f5ba146e7789a220a88"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "26f6e5073e308ace437ff19ed04d42ee"
server_key_iv: "c18d4da54d439f715ff235e66dad741c"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.228008 23664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:35.256189 23664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:35.260425 23664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:35.260743 23664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:35.261492 23664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:35.262650 23664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:35.262723 23664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.262787 23664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:35.262826 23664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.276916 23664 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33273
I20260926 13:52:35.277187 23793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33273 every 8 connection(s)
I20260926 13:52:35.278252 23664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:52:35.283514 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 23664
I20260926 13:52:35.283659 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:52:35.284057 11187 external_mini_cluster.cc:1428] Setting key 0cdccf2d141aa0e46955dbb4fa6768c4
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:35.301460 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:35.281111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37367 (local address 127.10.236.194:43177)
0926 13:52:35.281741 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:35.281802 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:35.283263 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:35.284147 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:35.284157 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:35.284532 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:35.285272 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:35.285296 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.286510 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.286513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:35.287993 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:35.288004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.291684 (+ 3680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.291689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:35.291952 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:35.292757 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:35.292785 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:35.294951 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:35.297519 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:35.297526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:35.297546 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:35.297997 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:35.298381 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:35.298384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:35.298387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:35.298600 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:35.299206 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:35.299214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:35.299637 (+ 423us) client_negotiation.cc:769] Sending connection context
0926 13:52:35.299870 (+ 233us) client_negotiation.cc:240] Negotiation successful
0926 13:52:35.300270 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":152,"threads_started":1}
I20260926 13:52:35.302518 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:35.282338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:43177 (local address 127.10.236.254:37367)
0926 13:52:35.282484 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:35.282488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:35.282506 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:52:35.283572 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:35.283577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:35.283649 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:35.283712 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:35.285457 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.286344 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.288144 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.288496 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.288580 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:35.295247 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:35.295251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:35.295253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:35.295283 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:35.297340 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:35.298145 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:35.298149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:35.298151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:35.298255 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:35.298735 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:35.298739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:35.298741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:35.299026 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:35.299140 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:35.302138 (+ 2998us) server_negotiation.cc:299] Negotiation successful
0926 13:52:35.302303 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:52:35.304816 23794 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37367
I20260926 13:52:35.305117 23794 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:35.305828 23794 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:35.307194 23444 ts_manager.cc:194] Registered new tserver with Master: ca116f6095244f5ba146e7789a220a88 (127.10.236.194:33273)
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:52:35.308103 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:43177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:52:35.332084 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37367
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:35.503110 23799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:35.503509 23799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:35.503577 23799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:35.509636 23799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:35.509724 23799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:35.509809 23799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:35.517697 23799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.236.212:51265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:35.520100 23799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:35.521206 23799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:35.533874 23804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:35.537629 23805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:35.537942 23807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:35.538216 23799 server_base.cc:1034] running on GCE node
I20260926 13:52:35.538848 23799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:35.539762 23799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:35.543053 23799 hybrid_clock.cc:648] HybridClock initialized: now 1790430755542967 us; error 92 us; skew 500 ppm
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:35.546903 23799 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:52:35.548103 23799 webserver.cc:460] Webserver started at http://127.10.236.195:36089/ using document root <none> and password file <none>
I20260926 13:52:35.548834 23799 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:35.548924 23799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:35.549172 23799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:35.566936 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "89740b4c497545dc90d7697d62c04a9e"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "5aad1fe47dc94a932a6f9981496497a6"
server_key_iv: "d8d9e0c4f959290f46c972a31bf8e58b"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.567734 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "89740b4c497545dc90d7697d62c04a9e"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "5aad1fe47dc94a932a6f9981496497a6"
server_key_iv: "d8d9e0c4f959290f46c972a31bf8e58b"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.587617 23799 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.003s sys 0.002s
I20260926 13:52:35.626849 23814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.628852 23799 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.002s sys 0.000s
I20260926 13:52:35.628998 23799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "89740b4c497545dc90d7697d62c04a9e"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "5aad1fe47dc94a932a6f9981496497a6"
server_key_iv: "d8d9e0c4f959290f46c972a31bf8e58b"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.629155 23799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:35.689059 23799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:35.692750 23799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:35.692940 23799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:35.693634 23799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:35.694666 23799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:35.694725 23799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.694777 23799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:35.694849 23799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.706220 23799 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:38465
I20260926 13:52:35.706271 23927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:38465 every 8 connection(s)
I20260926 13:52:35.707450 23799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:52:35.709928 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 23799
I20260926 13:52:35.710058 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:52:35.710376 11187 external_mini_cluster.cc:1428] Setting key 708735ce57e360b90045b3ab634ebd8c
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:35.724797 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:35.711232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:60983 (local address 127.10.236.254:37367)
0926 13:52:35.711349 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:35.711353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:35.711559 (+ 206us) server_negotiation.cc:407] Connection header received
0926 13:52:35.712764 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:35.712767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:35.712819 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:35.712867 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:35.714482 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.715378 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.717149 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.717505 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.717580 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:35.720619 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:35.720623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:35.720626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:35.720653 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:35.722425 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:35.723024 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:35.723028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:35.723030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:35.723114 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:35.723525 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:35.723529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:35.723531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:35.723742 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:35.723823 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:35.724522 (+ 699us) server_negotiation.cc:299] Negotiation successful
0926 13:52:35.724641 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:52:35.726573 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:35.710061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37367 (local address 127.10.236.195:60983)
0926 13:52:35.711380 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:35.711438 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:35.712526 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:35.713280 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:35.713289 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:35.713628 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:35.714277 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:35.714295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.715568 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.715571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:35.717012 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:35.717023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:35.717785 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:35.717789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:35.717991 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:35.718694 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:35.718715 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:35.720426 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:35.722556 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:35.722562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:35.722579 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:35.722911 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:35.723229 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:35.723231 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:35.723234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:35.723402 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:35.723890 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:35.723897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:35.724235 (+ 338us) client_negotiation.cc:769] Sending connection context
0926 13:52:35.725021 (+ 786us) client_negotiation.cc:240] Negotiation successful
0926 13:52:35.725466 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"thread_start_us":141,"threads_started":1}
I20260926 13:52:35.727825 23928 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37367
I20260926 13:52:35.728125 23928 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:35.728737 23928 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:35.729887 23444 ts_manager.cc:194] Registered new tserver with Master: 89740b4c497545dc90d7697d62c04a9e (127.10.236.195:38465)
I20260926 13:52:35.730641 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:60983
I20260926 13:52:35.738835 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:52:35.739609 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 23525
I20260926 13:52:35.745263 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 23664
I20260926 13:52:35.751775 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 23799
I20260926 13:52:35.758411 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 23412
I20260926 13:52:35.766515 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37367
--webserver_interface=127.10.236.254
--webserver_port=45271
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:37367
--ranger_config_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:35.879297 23931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:35.879657 23931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:35.879703 23931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:35.884032 23931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:35.884101 23931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:35.884121 23931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:35.884138 23931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:35.884153 23931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:35.889771 23931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:37367
--ranger_config_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:35.890795 23931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:35.891777 23931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:35.900192 23937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:35.900192 23939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:35.900254 23936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:35.900600 23931 server_base.cc:1034] running on GCE node
I20260926 13:52:35.901230 23931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:35.902403 23931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:35.903563 23931 hybrid_clock.cc:648] HybridClock initialized: now 1790430755903524 us; error 50 us; skew 500 ppm
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:35.906476 23931 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:35.907399 23931 webserver.cc:460] Webserver started at http://127.10.236.254:45271/ using document root <none> and password file <none>
I20260926 13:52:35.907980 23931 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:35.908077 23931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:52:35 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430755, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:35.974557 23931 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260926 13:52:35.977012 23947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:35.978475 23931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:52:35.978643 23931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "cd0f1bf5d2ba446ab78b2d4594851662"
format_stamp: "Formatted at 2026-09-26 13:52:33 on dist-test-slave-r34j"
server_key: "fd1cdb77b6b984a6526b6005934b4297"
server_key_iv: "16268e44b5eb3899d3bcd7b6aac28cfa"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:35.979234 23931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:36.005445 23931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:36.008671 23931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:36.008905 23931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:36.016335 23931 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:37367
I20260926 13:52:36.016342 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:37367 every 8 connection(s)
I20260926 13:52:36.017436 23931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:36.021737 24000 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:52:36.023412 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Bootstrap starting.
I20260926 13:52:36.024595 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 23931
I20260926 13:52:36.025143 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:34803
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:37367
--webserver_port=37211
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:36.041240 24000 log.cc:824] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:36.048107 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:52:36.048535 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Bootstrap complete.
I20260926 13:52:36.052918 24000 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:36.053328 24000 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0f1bf5d2ba446ab78b2d4594851662, State: Initialized, Role: FOLLOWER
I20260926 13:52:36.053846 24000 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:36.053963 24000 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:52:36.054054 24000 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:52:36.054147 24000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:52:36.056165 24000 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:36.056502 24000 leader_election.cc:302] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0f1bf5d2ba446ab78b2d4594851662; no voters:
I20260926 13:52:36.056802 24000 leader_election.cc:288] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:52:36.056949 24004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:52:36.057216 24004 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [term 2 LEADER]: Becoming Leader. State: Replica: cd0f1bf5d2ba446ab78b2d4594851662, State: Running, Role: LEADER
I20260926 13:52:36.057531 24004 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } }
I20260926 13:52:36.058056 24000 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:36.059013 24005 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } } }
I20260926 13:52:36.059154 24005 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:36.059769 24006 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0f1bf5d2ba446ab78b2d4594851662. Latest consensus state: current_term: 2 leader_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f1bf5d2ba446ab78b2d4594851662" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37367 } } }
I20260926 13:52:36.059944 24006 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:36.060252 24013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:36.060535 24000 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 13:52:36.066579 24013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:36.067548 24013 catalog_manager.cc:1259] Loaded cluster ID: b553352342a54330a51f2f155d9cd8be
I20260926 13:52:36.067611 24013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:52:36.071007 24013 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:52:36.071740 24013 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Loaded TSK: 0
I20260926 13:52:36.073271 24013 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:52:36.172940 24002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:36.173328 24002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:36.173411 24002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:36.179498 24002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:36.179585 24002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:36.179682 24002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:52:36.187619 24002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:34803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=37211
--tserver_master_addrs=127.10.236.254:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:36.188993 24002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:36.190335 24002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:36.203018 24040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:36.203003 24038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:36.203413 24037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:36.203586 24002 server_base.cc:1034] running on GCE node
I20260926 13:52:36.204170 24002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:36.205011 24002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:36.206239 24002 hybrid_clock.cc:648] HybridClock initialized: now 1790430756206202 us; error 50 us; skew 500 ppm
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:36.210156 24002 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:52:36.211346 24002 webserver.cc:460] Webserver started at http://127.10.236.193:37211/ using document root <none> and password file <none>
I20260926 13:52:36.212102 24002 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:36.212212 24002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:36.245715 24002 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260926 13:52:36.248842 24048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.250397 24002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:52:36.250517 24002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2ebb4e47adbc4b46a2ca564235d3f038"
format_stamp: "Formatted at 2026-09-26 13:52:34 on dist-test-slave-r34j"
server_key: "e77871580df44c9650e7af56ef48f642"
server_key_iv: "7c43edd4755a2c59a4dfd7586e6b2802"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:36.251225 24002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:36.278776 24002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:36.284504 24002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:36.284761 24002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:36.285539 24002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:36.286679 24002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:36.286749 24002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.286801 24002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:36.286823 24002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.298364 24002 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:34803
I20260926 13:52:36.298427 24161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:34803 every 8 connection(s)
I20260926 13:52:36.299875 24002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:52:36.304373 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24002
I20260926 13:52:36.304997 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:33273
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:37367
--webserver_port=37087
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:36.324045 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.302840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:56237 (local address 127.10.236.254:37367)
0926 13:52:36.303420 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:36.303441 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:36.303982 (+ 541us) server_negotiation.cc:407] Connection header received
0926 13:52:36.305438 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:36.305455 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:36.305891 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:36.306220 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:36.308010 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.309409 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.311082 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.311848 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.312038 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.318455 (+ 6417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:36.318463 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:36.318482 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:36.318517 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:36.320664 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.321236 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.321243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.321253 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.321362 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.321724 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.321727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.321733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.322096 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:36.322240 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:36.322798 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 13:52:36.323043 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":109,"threads_started":1}
I20260926 13:52:36.324057 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.303005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37367 (local address 127.10.236.193:56237)
0926 13:52:36.303776 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:36.303832 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:36.304910 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:36.306803 (+ 1893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:36.306812 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:36.307157 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:36.307812 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:36.307832 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.309574 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.309578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:36.310938 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:36.310949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.312302 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.312306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:36.312517 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.313195 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:36.313216 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:36.318220 (+ 5004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:36.320811 (+ 2591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:36.320817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:36.320832 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:36.321103 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:36.321463 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:36.321468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:36.321470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:36.321616 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:36.322219 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:36.322225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:36.322485 (+ 260us) client_negotiation.cc:769] Sending connection context
0926 13:52:36.322665 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 13:52:36.323005 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":142,"threads_started":1}
I20260926 13:52:36.326812 24162 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37367
I20260926 13:52:36.327069 24162 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.327682 24162 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
W20260926 13:52:36.329628 24162 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:37367 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 13:52:36.329767 24162 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.329950 24162 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:36.330711 24162 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.330880 24162 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
W20260926 13:52:36.331576 24162 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:52:36.469491 24166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:36.469807 24166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:36.469887 24166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:36.474324 24166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:36.474416 24166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:36.474504 24166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:52:36.480607 24166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:33273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=37087
--tserver_master_addrs=127.10.236.254:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:36.481830 24166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:36.483083 24166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:36.495195 24172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:36.495451 24171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:36.495673 24166 server_base.cc:1034] running on GCE node
W20260926 13:52:36.495962 24174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:36.496475 24166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:36.497256 24166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:36.498399 24166 hybrid_clock.cc:648] HybridClock initialized: now 1790430756498328 us; error 88 us; skew 500 ppm
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:36.502245 24166 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:52:36.503435 24166 webserver.cc:460] Webserver started at http://127.10.236.194:37087/ using document root <none> and password file <none>
I20260926 13:52:36.504174 24166 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:36.504272 24166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:36.534473 24166 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.003s sys 0.000s
I20260926 13:52:36.538110 24182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.539762 24166 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 13:52:36.539896 24166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "ca116f6095244f5ba146e7789a220a88"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "26f6e5073e308ace437ff19ed04d42ee"
server_key_iv: "c18d4da54d439f715ff235e66dad741c"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:36.540654 24166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:36.556969 24166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:36.561129 24166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:36.561347 24166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:36.562094 24166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:36.563130 24166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:36.563194 24166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.563253 24166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:36.563283 24166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.573328 24166 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33273
I20260926 13:52:36.573549 24296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33273 every 8 connection(s)
I20260926 13:52:36.574617 24166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:52:36.577373 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24166
I20260926 13:52:36.577824 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.236.212:51265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:38465
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:37367
--webserver_port=36089
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:36.597086 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.578504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:50283 (local address 127.10.236.254:37367)
0926 13:52:36.578653 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:36.578660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:36.579424 (+ 764us) server_negotiation.cc:407] Connection header received
0926 13:52:36.580783 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:36.580787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:36.580848 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:36.580906 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:36.582695 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.583533 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.585262 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.585590 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.585673 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.592471 (+ 6798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:36.592476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:36.592479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:36.592513 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:36.594331 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.595069 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.595073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.595075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.595171 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.595683 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.595687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.595689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.595947 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:36.596025 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:36.596771 (+ 746us) server_negotiation.cc:299] Negotiation successful
0926 13:52:36.596914 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:52:36.599042 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.577335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37367 (local address 127.10.236.194:50283)
0926 13:52:36.579247 (+ 1912us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:36.579304 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:36.580488 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:36.581348 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:36.581359 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:36.581742 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:36.582503 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:36.582525 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.583735 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.583739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:36.585119 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:36.585130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.585911 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.585914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:36.586110 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.587947 (+ 1837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:36.587968 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:36.592223 (+ 4255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:36.594494 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:36.594501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:36.594520 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:36.594926 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:36.595305 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:36.595308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:36.595310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:36.595555 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:36.596117 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:36.596124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:36.596470 (+ 346us) client_negotiation.cc:769] Sending connection context
0926 13:52:36.597347 (+ 877us) client_negotiation.cc:240] Negotiation successful
0926 13:52:36.597706 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1677,"thread_start_us":125,"threads_started":1}
I20260926 13:52:36.601151 24297 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37367
I20260926 13:52:36.601462 24297 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.602156 24297 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
W20260926 13:52:36.604023 24297 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:37367 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 13:52:36.604209 24297 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.604393 24297 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:36.605185 24297 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.605340 24297 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
W20260926 13:52:36.606029 24297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:52:36.762380 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:36.762801 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:36.762912 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:36.769098 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:36.769284 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:36.769418 24300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:52:36.777869 24300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.236.212:51265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:38465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=36089
--tserver_master_addrs=127.10.236.254:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:36.779336 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:36.780712 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:36.793614 24308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:36.804275 24306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:36.804488 24305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:36.805023 24300 server_base.cc:1034] running on GCE node
I20260926 13:52:36.805698 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:36.806694 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:36.807919 24300 hybrid_clock.cc:648] HybridClock initialized: now 1790430756807895 us; error 37 us; skew 500 ppm
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:36.812214 24300 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:52:36.813416 24300 webserver.cc:460] Webserver started at http://127.10.236.195:36089/ using document root <none> and password file <none>
I20260926 13:52:36.814145 24300 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:36.814248 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:36.838873 24300 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 13:52:36.842410 24316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.844164 24300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:52:36.844374 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "89740b4c497545dc90d7697d62c04a9e"
format_stamp: "Formatted at 2026-09-26 13:52:35 on dist-test-slave-r34j"
server_key: "5aad1fe47dc94a932a6f9981496497a6"
server_key_iv: "d8d9e0c4f959290f46c972a31bf8e58b"
server_key_version: "kuduclusterkey@0"
I20260926 13:52:36.845225 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:36.873479 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:36.877975 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:36.878294 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:36.879135 24300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:36.880376 24300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:36.880488 24300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.880599 24300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:36.880673 24300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:36.892540 24300 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:38465
I20260926 13:52:36.892721 24429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:38465 every 8 connection(s)
I20260926 13:52:36.894079 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:52:36.896848 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24300
Sep 26 13:52:36 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430756, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:36.910137 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.897510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.10.236.254:37367)
0926 13:52:36.898449 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:36.898454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:36.899859 (+ 1405us) server_negotiation.cc:407] Connection header received
0926 13:52:36.899904 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:36.899906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:36.899952 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:36.900030 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:36.901025 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.901882 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.903047 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.903371 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.903465 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.905591 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:36.905607 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:36.905610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:36.905634 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:36.907692 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.908408 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.908411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.908413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.908498 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.908790 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.908793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.908795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.909014 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:36.909210 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:36.909834 (+ 624us) server_negotiation.cc:299] Negotiation successful
0926 13:52:36.909955 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":862,"spinlock_wait_cycles":2304,"thread_start_us":84,"threads_started":1}
I20260926 13:52:36.911428 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.897457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:54561 (local address 127.10.236.254:37367)
0926 13:52:36.897627 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:36.897632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:36.897964 (+ 332us) server_negotiation.cc:407] Connection header received
0926 13:52:36.899350 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:36.899354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:36.899413 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:36.899542 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:36.901137 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.901910 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.903648 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.903989 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.904065 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.907128 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:36.907132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:36.907135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:36.907182 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:36.908825 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.909444 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.909447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.909449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.909530 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:36.909943 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:36.909946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:36.909948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:36.910143 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:36.910210 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:36.911170 (+ 960us) server_negotiation.cc:299] Negotiation successful
0926 13:52:36.911284 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:52:36.913326 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:36.897127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37367 (local address 127.10.236.195:54561)
0926 13:52:36.897809 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:36.897864 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:36.899089 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:36.899872 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:36.899881 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:36.900256 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:36.900951 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:36.900980 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.902079 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.902082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:36.903475 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:36.903525 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:36.904265 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:36.904270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:36.904513 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:36.905085 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:36.905108 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:36.906926 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:36.908956 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:36.908961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:36.908979 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:36.909325 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:36.909638 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:36.909641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:36.909643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:36.909830 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:36.910541 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:36.910548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:36.910881 (+ 333us) client_negotiation.cc:769] Sending connection context
0926 13:52:36.911803 (+ 922us) client_negotiation.cc:240] Negotiation successful
0926 13:52:36.912157 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":140,"threads_started":1}
I20260926 13:52:36.914847 24430 heartbeater.cc:347] Connected to a master server at 127.10.236.254:37367
I20260926 13:52:36.915103 24430 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.915736 24430 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
W20260926 13:52:36.917282 24430 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:37367 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 13:52:36.917392 24430 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.917539 24430 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:36.918233 24430 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:36.918372 24430 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
W20260926 13:52:36.919126 24430 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 13:52:37 dist-test-slave-r34j krb5kdc[22968](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790430757, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:37.331876 24162 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:37.332244 24162 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:37.333598 23963 ts_manager.cc:194] Registered new tserver with Master: 2ebb4e47adbc4b46a2ca564235d3f038 (127.10.236.193:34803)
I20260926 13:52:37.335255 23963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:56237
I20260926 13:52:37.606285 24297 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:37.606619 24297 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:37.607620 23963 ts_manager.cc:194] Registered new tserver with Master: ca116f6095244f5ba146e7789a220a88 (127.10.236.194:33273)
I20260926 13:52:37.608417 23963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:50283
Sep 26 13:52:37 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430757, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:37.919377 24430 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:37.919728 24430 heartbeater.cc:511] Master 127.10.236.254:37367 requested a full tablet report, sending...
I20260926 13:52:37.920542 23963 ts_manager.cc:194] Registered new tserver with Master: 89740b4c497545dc90d7697d62c04a9e (127.10.236.195:38465)
I20260926 13:52:37.921384 23963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:54561
I20260926 13:52:37.929487 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:52:37.946621 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.931366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.10.236.254:37367)
0926 13:52:37.931708 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:37.931713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:37.932270 (+ 557us) server_negotiation.cc:407] Connection header received
0926 13:52:37.932317 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:37.932320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:37.932371 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:37.932451 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:37.933438 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.934311 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.935549 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.935904 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.936015 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.941555 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:37.941560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:37.941563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:37.941600 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:37.943431 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:37.944016 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:37.944020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:37.944022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:37.944108 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:37.944672 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:37.944676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:37.944678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:37.944927 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:37.945049 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:37.945901 (+ 852us) server_negotiation.cc:299] Negotiation successful
0926 13:52:37.946086 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":142,"threads_started":1}
Sep 26 13:52:37 dist-test-slave-r34j krb5kdc[22968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430757, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for HTTP/127.10.236.212@KRBTEST.COM
I20260926 13:52:37.951776 23963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:52:37.957072 23963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:52:37.988170 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.979211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:34803 (local address 127.0.0.1:39406)
0926 13:52:37.980680 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:37.980708 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:37.980807 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:37.982311 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:37.982315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:37.982334 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:37.982422 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:37.982428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.983990 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.983993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:37.985355 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:37.985367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.987575 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.987579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:37.987705 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.987753 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 13:52:37.987870 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 13:52:37.988002 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"thread_start_us":87,"threads_started":1}
I20260926 13:52:37.988778 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.979642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.10.236.193:34803)
0926 13:52:37.980556 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:37.980564 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:37.980896 (+ 332us) server_negotiation.cc:407] Connection header received
0926 13:52:37.980980 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:37.980986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:37.981235 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:37.981383 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:37.982579 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.983869 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.985493 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.986293 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.986408 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.986545 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:37.988525 (+ 1980us) server_negotiation.cc:299] Negotiation successful
0926 13:52:37.988645 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":87,"threads_started":1}
I20260926 13:52:37.989971 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.977701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:38465 (local address 127.0.0.1:38700)
0926 13:52:37.980247 (+ 2546us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:37.980260 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:37.980335 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:37.982132 (+ 1797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:37.982139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:37.982166 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:37.982332 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:37.983191 (+ 859us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.985225 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.985228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:37.987815 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:37.987827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.989564 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.989569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:37.989667 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.989691 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 13:52:37.989764 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:52:37.989838 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2477,"thread_start_us":107,"threads_started":1}
I20260926 13:52:37.992200 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.978174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.10.236.195:38465)
0926 13:52:37.981669 (+ 3495us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:37.981676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:37.981698 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:52:37.981769 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:37.981774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:37.981955 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:37.982123 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:37.983793 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.985073 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.988344 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.989245 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.989371 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.989500 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:37.991927 (+ 2427us) server_negotiation.cc:299] Negotiation successful
0926 13:52:37.992045 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3387,"thread_start_us":105,"threads_started":1}
I20260926 13:52:37.992672 24088 tablet_service.cc:1467] Processing CreateTablet for tablet b375978cfd9a44348d38e20992a78904 (DEFAULT_TABLE table=test-table [id=0cc0a79151904ea18f5e8805f9ced133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:37.993980 24088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b375978cfd9a44348d38e20992a78904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:37.995191 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.974537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33273 (local address 127.0.0.1:47858)
0926 13:52:37.979825 (+ 5288us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:37.979898 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:37.980037 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:37.987328 (+ 7291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:37.987331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:37.987355 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:37.987470 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:37.987476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.991586 (+ 4110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.991590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:37.992711 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:37.992723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.994647 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.994652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:37.994847 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.994877 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 13:52:37.994941 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 13:52:37.995013 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5048,"thread_start_us":142,"threads_started":1}
I20260926 13:52:37.996223 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:37.975318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.10.236.194:33273)
0926 13:52:37.986759 (+ 11441us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:37.986766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:37.986790 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:52:37.986870 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:37.986875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:37.987108 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:37.987381 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:37.990300 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.991415 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.993714 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:37.994506 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:37.995760 (+ 1254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:37.995920 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:37.995975 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 13:52:37.996076 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11321,"thread_start_us":114,"threads_started":1}
I20260926 13:52:37.995971 24364 tablet_service.cc:1467] Processing CreateTablet for tablet b375978cfd9a44348d38e20992a78904 (DEFAULT_TABLE table=test-table [id=0cc0a79151904ea18f5e8805f9ced133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:37.998345 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b375978cfd9a44348d38e20992a78904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:37.999730 24230 tablet_service.cc:1467] Processing CreateTablet for tablet b375978cfd9a44348d38e20992a78904 (DEFAULT_TABLE table=test-table [id=0cc0a79151904ea18f5e8805f9ced133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:38.001150 24230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b375978cfd9a44348d38e20992a78904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:38.011023 24465 tablet_bootstrap.cc:492] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Bootstrap starting.
I20260926 13:52:38.011510 24466 tablet_bootstrap.cc:492] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Bootstrap starting.
I20260926 13:52:38.011817 24464 tablet_bootstrap.cc:492] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: Bootstrap starting.
I20260926 13:52:38.014524 24466 tablet_bootstrap.cc:654] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:38.014524 24464 tablet_bootstrap.cc:654] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:38.015527 24464 log.cc:824] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:38.017246 24465 tablet_bootstrap.cc:654] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:38.018211 24466 log.cc:824] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:38.018738 24464 tablet_bootstrap.cc:492] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: No bootstrap required, opened a new log
I20260926 13:52:38.018952 24464 ts_tablet_manager.cc:1397] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:52:38.019253 24465 log.cc:824] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:38.022357 24466 tablet_bootstrap.cc:492] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: No bootstrap required, opened a new log
I20260926 13:52:38.022569 24466 ts_tablet_manager.cc:1397] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260926 13:52:38.023250 24464 raft_consensus.cc:348] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.023818 24464 raft_consensus.cc:374] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:38.023921 24464 raft_consensus.cc:729] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebb4e47adbc4b46a2ca564235d3f038, State: Initialized, Role: FOLLOWER
I20260926 13:52:38.024549 24464 consensus_queue.cc:260] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.025815 24464 ts_tablet_manager.cc:1428] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:52:38.026240 24162 heartbeater.cc:503] Master 127.10.236.254:37367 was elected leader, sending a full tablet report...
I20260926 13:52:38.026604 24466 raft_consensus.cc:348] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.026868 24466 raft_consensus.cc:374] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:38.026926 24466 raft_consensus.cc:729] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca116f6095244f5ba146e7789a220a88, State: Initialized, Role: FOLLOWER
I20260926 13:52:38.027422 24466 consensus_queue.cc:260] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.028551 24466 ts_tablet_manager.cc:1428] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:52:38.028997 24297 heartbeater.cc:503] Master 127.10.236.254:37367 was elected leader, sending a full tablet report...
I20260926 13:52:38.033599 24465 tablet_bootstrap.cc:492] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: No bootstrap required, opened a new log
I20260926 13:52:38.033844 24465 ts_tablet_manager.cc:1397] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260926 13:52:38.037886 24465 raft_consensus.cc:348] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.038259 24465 raft_consensus.cc:374] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:38.038347 24465 raft_consensus.cc:729] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89740b4c497545dc90d7697d62c04a9e, State: Initialized, Role: FOLLOWER
I20260926 13:52:38.038909 24465 consensus_queue.cc:260] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.040349 24465 ts_tablet_manager.cc:1428] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260926 13:52:38.040750 24430 heartbeater.cc:503] Master 127.10.236.254:37367 was elected leader, sending a full tablet report...
W20260926 13:52:38.052219 24163 tablet.cc:2369] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:52:38.090376 24298 tablet.cc:2369] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:52:38.146490 24431 tablet.cc:2369] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:52:38.169741 24472 raft_consensus.cc:482] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:38.169976 24472 raft_consensus.cc:504] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.171341 24472 leader_election.cc:288] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca116f6095244f5ba146e7789a220a88 (127.10.236.194:33273), 2ebb4e47adbc4b46a2ca564235d3f038 (127.10.236.193:34803)
I20260926 13:52:38.178023 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.172316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:58627 (local address 127.10.236.194:33273)
0926 13:52:38.172439 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:38.172443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:38.173613 (+ 1170us) server_negotiation.cc:407] Connection header received
0926 13:52:38.173828 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:38.173831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:38.173879 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:38.173926 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:38.174352 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.175147 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.176740 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.177315 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.177410 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:38.177463 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:38.177802 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 13:52:38.177883 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:52:38.178812 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.171825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33273 (local address 127.10.236.195:58627)
0926 13:52:38.173507 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:38.173522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:38.173704 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:38.174064 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:38.174066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:38.174083 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:38.174225 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:38.174231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.175293 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.175296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:38.176596 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:38.176607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.177567 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.177570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:38.177669 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:38.177691 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:52:38.178568 (+ 877us) client_negotiation.cc:240] Negotiation successful
0926 13:52:38.178651 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1607,"thread_start_us":144,"threads_started":1}
I20260926 13:52:38.179735 24250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b375978cfd9a44348d38e20992a78904" candidate_uuid: "89740b4c497545dc90d7697d62c04a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca116f6095244f5ba146e7789a220a88" is_pre_election: true
I20260926 13:52:38.179908 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.173197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:58039 (local address 127.10.236.193:34803)
0926 13:52:38.173317 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:38.173322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:38.173726 (+ 404us) server_negotiation.cc:407] Connection header received
0926 13:52:38.174251 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:38.174254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:38.174310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:38.174363 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:38.176275 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.177051 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.178554 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.179166 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.179262 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:38.179319 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:38.179684 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 13:52:38.179782 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:52:38.180087 24250 raft_consensus.cc:2454] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89740b4c497545dc90d7697d62c04a9e in term 0.
I20260926 13:52:38.180335 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.172821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:34803 (local address 127.10.236.195:58039)
0926 13:52:38.173572 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:38.173591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:38.173812 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:38.175962 (+ 2150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:38.175965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:38.175986 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:38.176130 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:38.176136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.177201 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.177204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:38.178394 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:38.178404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.179416 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.179421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:38.179538 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:38.179559 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 13:52:38.180112 (+ 553us) client_negotiation.cc:240] Negotiation successful
0926 13:52:38.180190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":106,"threads_started":1}
I20260926 13:52:38.180923 24317 leader_election.cc:302] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89740b4c497545dc90d7697d62c04a9e, ca116f6095244f5ba146e7789a220a88; no voters:
I20260926 13:52:38.180962 24098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b375978cfd9a44348d38e20992a78904" candidate_uuid: "89740b4c497545dc90d7697d62c04a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" is_pre_election: true
I20260926 13:52:38.181241 24098 raft_consensus.cc:2454] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89740b4c497545dc90d7697d62c04a9e in term 0.
I20260926 13:52:38.181252 24472 raft_consensus.cc:2781] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:52:38.181324 24472 raft_consensus.cc:482] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:52:38.181367 24472 raft_consensus.cc:3037] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:38.182732 24472 raft_consensus.cc:504] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.183108 24472 leader_election.cc:288] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [CANDIDATE]: Term 1 election: Requested vote from peers ca116f6095244f5ba146e7789a220a88 (127.10.236.194:33273), 2ebb4e47adbc4b46a2ca564235d3f038 (127.10.236.193:34803)
I20260926 13:52:38.183725 24098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b375978cfd9a44348d38e20992a78904" candidate_uuid: "89740b4c497545dc90d7697d62c04a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebb4e47adbc4b46a2ca564235d3f038"
I20260926 13:52:38.183844 24098 raft_consensus.cc:3037] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:38.185124 24098 raft_consensus.cc:2454] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89740b4c497545dc90d7697d62c04a9e in term 1.
I20260926 13:52:38.185540 24320 leader_election.cc:302] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ebb4e47adbc4b46a2ca564235d3f038, 89740b4c497545dc90d7697d62c04a9e; no voters:
I20260926 13:52:38.186038 24472 raft_consensus.cc:2781] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:38.186081 24250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b375978cfd9a44348d38e20992a78904" candidate_uuid: "89740b4c497545dc90d7697d62c04a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca116f6095244f5ba146e7789a220a88"
I20260926 13:52:38.186208 24250 raft_consensus.cc:3037] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:38.186287 24472 raft_consensus.cc:686] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 1 LEADER]: Becoming Leader. State: Replica: 89740b4c497545dc90d7697d62c04a9e, State: Running, Role: LEADER
I20260926 13:52:38.186662 24472 consensus_queue.cc:237] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } }
I20260926 13:52:38.187505 24250 raft_consensus.cc:2454] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89740b4c497545dc90d7697d62c04a9e in term 1.
I20260926 13:52:38.190128 23963 catalog_manager.cc:5495] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 89740b4c497545dc90d7697d62c04a9e (127.10.236.195). New cstate: current_term: 1 leader_uuid: "89740b4c497545dc90d7697d62c04a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89740b4c497545dc90d7697d62c04a9e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 38465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 } health_report { overall_health: UNKNOWN } } }
I20260926 13:52:38.218395 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.214227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.10.236.195:38465)
0926 13:52:38.214407 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:38.214413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:38.214621 (+ 208us) server_negotiation.cc:407] Connection header received
0926 13:52:38.214850 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:38.214854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:38.214921 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:38.215039 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:52:38.215290 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.215905 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.216833 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:38.217089 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:38.217201 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:38.217515 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:52:38.217952 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:52:38.218087 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:38.218158 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 13:52:38.218222 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 13:52:38.224579 24250 raft_consensus.cc:1261] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 1 FOLLOWER]: Refusing update from remote peer 89740b4c497545dc90d7697d62c04a9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:38.224579 24098 raft_consensus.cc:1261] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 1 FOLLOWER]: Refusing update from remote peer 89740b4c497545dc90d7697d62c04a9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:38.225483 24472 consensus_queue.cc:1035] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca116f6095244f5ba146e7789a220a88" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:38.226007 24476 consensus_queue.cc:1035] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebb4e47adbc4b46a2ca564235d3f038" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:38.237344 24479 mvcc.cc:205] Tried to move back new op lower bound from 7333604385680248832 to 7333604385536606208. Current Snapshot: MvccSnapshot[applied={T|T < 7333604385680248832}]
I20260926 13:52:38.239486 24480 mvcc.cc:205] Tried to move back new op lower bound from 7333604385680248832 to 7333604385536606208. Current Snapshot: MvccSnapshot[applied={T|T < 7333604385680248832}]
I20260926 13:52:38.247916 24481 mvcc.cc:205] Tried to move back new op lower bound from 7333604385680248832 to 7333604385536606208. Current Snapshot: MvccSnapshot[applied={T|T < 7333604385680248832}]
I20260926 13:52:38.254716 23964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:52:38.254921 23964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:52:38.258038 23964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd0f1bf5d2ba446ab78b2d4594851662: Sending DeleteTablet for 3 replicas of tablet b375978cfd9a44348d38e20992a78904
I20260926 13:52:38.259231 24088 tablet_service.cc:1514] Processing DeleteTablet for tablet b375978cfd9a44348d38e20992a78904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:38 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:39406
I20260926 13:52:38.260013 24230 tablet_service.cc:1514] Processing DeleteTablet for tablet b375978cfd9a44348d38e20992a78904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:38 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:47858
I20260926 13:52:38.260479 24488 tablet_replica.cc:330] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038: stopping tablet replica
I20260926 13:52:38.260814 24488 raft_consensus.cc:2229] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:38.261199 24488 raft_consensus.cc:2258] T b375978cfd9a44348d38e20992a78904 P 2ebb4e47adbc4b46a2ca564235d3f038 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:38.261193 24364 tablet_service.cc:1514] Processing DeleteTablet for tablet b375978cfd9a44348d38e20992a78904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:38 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:38700
I20260926 13:52:38.262605 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24002
I20260926 13:52:38.269659 24490 tablet_replica.cc:330] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: stopping tablet replica
I20260926 13:52:38.270134 24490 raft_consensus.cc:2229] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:38.270593 24490 raft_consensus.cc:2258] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:38.271581 24491 tablet_replica.cc:330] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: stopping tablet replica
I20260926 13:52:38.272415 24490 ts_tablet_manager.cc:1905] T b375978cfd9a44348d38e20992a78904 P ca116f6095244f5ba146e7789a220a88: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:52:38.272727 23951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:34803 (error 108)
I20260926 13:52:38.274016 24491 raft_consensus.cc:2229] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:52:38.274596 24491 raft_consensus.cc:2258] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:38.275472 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24166
I20260926 13:52:38.277406 24491 ts_tablet_manager.cc:1905] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:52:38.280985 24491 ts_tablet_manager.cc:1918] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:52:38.281159 24491 log.cc:1196] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/b375978cfd9a44348d38e20992a78904
I20260926 13:52:38.281577 24491 ts_tablet_manager.cc:1939] T b375978cfd9a44348d38e20992a78904 P 89740b4c497545dc90d7697d62c04a9e: Deleting consensus metadata
I20260926 13:52:38.282262 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.281822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:34803 (local address 127.0.0.1:39410)
0926 13:52:38.282092 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:38.282171 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:34803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
W20260926 13:52:38.282612 23951 catalog_manager.cc:4584] TS 2ebb4e47adbc4b46a2ca564235d3f038 (127.10.236.193:34803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b375978cfd9a44348d38e20992a78904: Network error: Client connection negotiation failed: client connection to 127.10.236.193:34803: connect: Connection refused (error 111)
I20260926 13:52:38.282950 23948 catalog_manager.cc:4841] TS 89740b4c497545dc90d7697d62c04a9e (127.10.236.195:38465): tablet b375978cfd9a44348d38e20992a78904 (table test-table [id=0cc0a79151904ea18f5e8805f9ced133]) successfully deleted
I20260926 13:52:38.286767 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:38.286533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33273 (local address 127.0.0.1:47866)
0926 13:52:38.286631 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:38.286696 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.194:33273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 13:52:38.287138 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24300
I20260926 13:52:38.294121 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 23931
I20260926 13:52:38.302933 11187 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 13:52:38.638413 11187 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 13:52:38.638554 11187 mini_ranger.cc:66] Stopping Ranger...
I20260926 13:52:38.984712 11187 mini_ranger.cc:68] Stopped Ranger
2026-09-26 13:52:38.984 UTC [22998] LOG: received smart shutdown request
2026-09-26 13:52:38.989 UTC [22998] LOG: background worker "logical replication launcher" (PID 23007) exited with exit code 1
2026-09-26 13:52:38.989 UTC [23002] LOG: shutting down
2026-09-26 13:52:39.198 UTC [22998] LOG: database system is shut down
2026-09-26T13:52:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51982 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:52:39 dist-test-slave-r34j krb5kdc[24507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:52:39 dist-test-slave-r34j krb5kdc[24507](info): set up 2 sockets
Sep 26 13:52:39 dist-test-slave-r34j krb5kdc[24507](info): commencing operation
krb5kdc: starting...
W20260926 13:52:40.411671 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:52:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:52:40.543370 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:38517
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:36339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:38517,127.10.236.253:36553,127.10.236.252:35527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:40.667609 24520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:40.667956 24520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:40.668048 24520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:40.672394 24520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:40.672497 24520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:40.672534 24520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:40.672574 24520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:40.672616 24520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:40.678380 24520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:36339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:38517,127.10.236.253:36553,127.10.236.252:35527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:38517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:40.679625 24520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:40.680713 24520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:40.689863 24525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:40.689800 24526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:40.689821 24528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:40.690044 24520 server_base.cc:1034] running on GCE node
I20260926 13:52:40.690775 24520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:40.691970 24520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:40.693188 24520 hybrid_clock.cc:648] HybridClock initialized: now 1790430760693148 us; error 46 us; skew 500 ppm
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:40.696321 24520 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:40.697314 24520 webserver.cc:460] Webserver started at http://127.10.236.254:46277/ using document root <none> and password file <none>
I20260926 13:52:40.697907 24520 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:40.697971 24520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:40.698206 24520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:40.700667 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "cc8a579169db489fbfccf7281b96622c"
format_stamp: "Formatted at 2026-09-26 13:52:40 on dist-test-slave-r34j"
server_key: "9266dbb310efbf911f382a8a6496f543"
server_key_iv: "dd17563ed00f9e6e00041408eacf6039"
server_key_version: "encryptionkey@0"
I20260926 13:52:40.701314 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "cc8a579169db489fbfccf7281b96622c"
format_stamp: "Formatted at 2026-09-26 13:52:40 on dist-test-slave-r34j"
server_key: "9266dbb310efbf911f382a8a6496f543"
server_key_iv: "dd17563ed00f9e6e00041408eacf6039"
server_key_version: "encryptionkey@0"
I20260926 13:52:40.705348 24520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:52:40.707993 24535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:40.709331 24520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:40.709487 24520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "cc8a579169db489fbfccf7281b96622c"
format_stamp: "Formatted at 2026-09-26 13:52:40 on dist-test-slave-r34j"
server_key: "9266dbb310efbf911f382a8a6496f543"
server_key_iv: "dd17563ed00f9e6e00041408eacf6039"
server_key_version: "encryptionkey@0"
I20260926 13:52:40.709613 24520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:40.724767 24520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:40.728999 24520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:40.729403 24520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:40.737355 24520 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:38517
I20260926 13:52:40.737967 24587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:38517 every 8 connection(s)
I20260926 13:52:40.738474 24520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:40.740742 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24520
I20260926 13:52:40.740895 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:52:40.741199 11187 external_mini_cluster.cc:1428] Setting key b84cf1993ac595bb351200a04ebcdf69
I20260926 13:52:40.741879 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:40.746320 24588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.236.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.253@KRBTEST.COM" created.
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:40.762773 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.748852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38517 (local address 127.0.0.1:58152)
0926 13:52:40.749440 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.749489 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:40.750397 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:40.751088 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:40.751093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:40.751553 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:40.751829 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.751843 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.753333 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.753336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.754091 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.754101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.755082 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.755085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.755192 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.755894 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:40.755911 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:40.757454 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:40.759589 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:40.759598 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:40.759608 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:40.759845 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:40.760174 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:40.760178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:40.760181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:40.760298 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:40.760911 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:40.760918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:40.761079 (+ 161us) client_negotiation.cc:769] Sending connection context
0926 13:52:40.761267 (+ 188us) client_negotiation.cc:240] Negotiation successful
0926 13:52:40.761680 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":107,"spinlock_wait_cycles":24960,"thread_start_us":154,"threads_started":1}
I20260926 13:52:40.762773 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.748852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.10.236.254:38517)
0926 13:52:40.749301 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:40.749319 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:40.749597 (+ 278us) server_negotiation.cc:407] Connection header received
0926 13:52:40.750708 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:40.750719 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:40.750904 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:40.751092 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:40.751978 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.753164 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.754206 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.754853 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.755047 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.757613 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:40.757620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:40.757641 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:40.757662 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:40.759413 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:40.759962 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:40.759969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:40.759971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:40.760081 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:40.760398 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:40.760401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:40.760403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:40.760798 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:40.760941 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:40.761355 (+ 414us) server_negotiation.cc:299] Negotiation successful
0926 13:52:40.761630 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":301,"thread_start_us":152,"threads_started":1}
I20260926 13:52:40.765391 24588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:40.766304 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.765984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45790)
0926 13:52:40.766123 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.766230 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 13:52:40.766505 24537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
I20260926 13:52:40.767591 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.767299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45806)
0926 13:52:40.767432 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.767489 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 13:52:40.767735 24588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:36553: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab.
Entry for principal kudu/127.10.236.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab.
I20260926 13:52:40.771775 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:36553
--webserver_interface=127.10.236.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab
--principal=kudu/127.10.236.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:36339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:38517,127.10.236.253:36553,127.10.236.252:35527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:40.817008 24588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } attempt: 1
I20260926 13:52:40.817924 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.817601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45812)
0926 13:52:40.817777 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.817843 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 13:52:40.818809 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.818589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45814)
0926 13:52:40.818704 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.818752 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:52:40.819005 24588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:36553: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
I20260926 13:52:40.887207 24588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } attempt: 2
I20260926 13:52:40.888170 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.887825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45828)
0926 13:52:40.888012 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.888089 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 13:52:40.888965 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.888715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45842)
0926 13:52:40.888844 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.888908 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 13:52:40.889153 24588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:36553: Network error: Client connection negotiation failed: client connection to 127.10.236.253:36553: connect: Connection refused (error 111)
W20260926 13:52:40.889585 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:40.889846 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:40.889941 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:40.894165 24595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:40.894235 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:40.894256 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:40.894274 24595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:40.894290 24595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:40.899923 24595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:36339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:38517,127.10.236.253:36553,127.10.236.252:35527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab
--principal=kudu/127.10.236.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:36553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.236.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:40.901101 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:40.902175 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:52:40.911266 24595 server_base.cc:1034] running on GCE node
W20260926 13:52:40.911190 24603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:40.911170 24600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:40.911170 24601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:40.912078 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:40.913379 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:40.914580 24595 hybrid_clock.cc:648] HybridClock initialized: now 1790430760914555 us; error 43 us; skew 500 ppm
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:40.918031 24595 init.cc:377] Logged in from keytab as kudu/127.10.236.253@KRBTEST.COM (short username kudu)
I20260926 13:52:40.919093 24595 webserver.cc:460] Webserver started at http://127.10.236.253:44275/ using document root <none> and password file <none>
I20260926 13:52:40.919819 24595 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:40.919919 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:40.920181 24595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:40.922480 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data/instance:
uuid: "d3e885f840744b54a349c69c5ce92264"
format_stamp: "Formatted at 2026-09-26 13:52:40 on dist-test-slave-r34j"
server_key: "8504151450deae653d7a226cf4e71ad9"
server_key_iv: "cc41739ce4c51808cb54732fa1d675bd"
server_key_version: "encryptionkey@0"
I20260926 13:52:40.923103 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal/instance:
uuid: "d3e885f840744b54a349c69c5ce92264"
format_stamp: "Formatted at 2026-09-26 13:52:40 on dist-test-slave-r34j"
server_key: "8504151450deae653d7a226cf4e71ad9"
server_key_iv: "cc41739ce4c51808cb54732fa1d675bd"
server_key_version: "encryptionkey@0"
I20260926 13:52:40.927217 24595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260926 13:52:40.930194 24610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:40.931725 24595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:40.931870 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal
uuid: "d3e885f840744b54a349c69c5ce92264"
format_stamp: "Formatted at 2026-09-26 13:52:40 on dist-test-slave-r34j"
server_key: "8504151450deae653d7a226cf4e71ad9"
server_key_iv: "cc41739ce4c51808cb54732fa1d675bd"
server_key_version: "encryptionkey@0"
I20260926 13:52:40.931991 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:40.952090 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:40.955292 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:40.955559 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:40.964825 24595 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.253:36553
I20260926 13:52:40.964905 24662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.253:36553 every 8 connection(s)
I20260926 13:52:40.965920 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
I20260926 13:52:40.968948 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24595
I20260926 13:52:40.968990 24663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:40.969168 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-1/wal/instance
I20260926 13:52:40.969439 11187 external_mini_cluster.cc:1428] Setting key af2e3f3e7af4844f17500846decd30f3
I20260926 13:52:40.974682 24663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.236.252@KRBTEST.COM; defaulting to no policy
I20260926 13:52:40.980336 24588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.252@KRBTEST.COM" created.
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:52:40.991968 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.976782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.10.236.254:38517)
0926 13:52:40.976980 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:40.976987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:40.977868 (+ 881us) server_negotiation.cc:407] Connection header received
0926 13:52:40.978861 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:40.978864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:40.978917 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:40.979009 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:40.980519 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.981495 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.982926 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.983300 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.983418 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.986839 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:40.986843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:40.986846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:40.986872 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:40.989394 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:40.989946 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:40.989952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:40.989955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:40.990049 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:40.990557 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:40.990561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:40.990563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:40.990838 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:40.990933 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:40.991644 (+ 711us) server_negotiation.cc:299] Negotiation successful
0926 13:52:40.991785 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"spinlock_wait_cycles":6016}
I20260926 13:52:40.992830 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.980823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45844)
0926 13:52:40.980998 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.981015 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:40.981103 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:40.981936 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:40.981939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:40.982127 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:40.982237 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.982244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.983716 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.983721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.984564 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.984574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.985452 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.985456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.985581 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.986109 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:40.986127 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:40.987730 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:40.990895 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:40.990899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:40.990901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:40.991147 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:40.991466 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:40.991469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:40.991471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:40.991607 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:40.992323 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:40.992325 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:40.992405 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 13:52:40.992521 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 13:52:40.992657 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"spinlock_wait_cycles":1280}
I20260926 13:52:40.992887 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.977156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38517 (local address 127.0.0.1:58156)
0926 13:52:40.977679 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.977733 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:40.978645 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:40.979210 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:40.979217 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:40.979729 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:40.980248 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.980262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.981646 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.981653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.982804 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.982813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.983463 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.983467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.983732 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.984422 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:40.984447 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:40.986677 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:40.989517 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:40.989529 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:40.989542 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:40.989837 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:40.990181 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:40.990186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:40.990189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:40.990427 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:40.990928 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:40.990935 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:40.991253 (+ 318us) client_negotiation.cc:769] Sending connection context
0926 13:52:40.991558 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 13:52:40.991913 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":3,"thread_start_us":149,"threads_started":1}
I20260926 13:52:40.993054 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.980941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.10.236.253:36553)
0926 13:52:40.981424 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:40.981434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:40.981459 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:52:40.981502 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:40.981508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:40.981752 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:40.981925 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:40.982367 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.983601 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.984693 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.985313 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.985436 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.988174 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:40.988184 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:40.988203 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:40.988237 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:40.990770 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:40.991269 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:40.991276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:40.991277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:40.991358 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:40.991711 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:40.991717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:40.991722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:40.991968 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:40.992162 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:40.992611 (+ 449us) server_negotiation.cc:299] Negotiation successful
0926 13:52:40.992756 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":158,"threads_started":1}
I20260926 13:52:40.994390 24663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:40.995146 24588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:40.995887 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.995654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50286)
0926 13:52:40.995783 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.995830 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:52:40.996646 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.996413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50298)
0926 13:52:40.996547 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.996598 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260926 13:52:40.996857 24588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:35527: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Sep 26 13:52:40 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab.
Entry for principal kudu/127.10.236.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab.
I20260926 13:52:41.003024 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:35527
--webserver_interface=127.10.236.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab
--principal=kudu/127.10.236.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:36339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:38517,127.10.236.253:36553,127.10.236.252:35527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:41.003697 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.994952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45848)
0926 13:52:40.995240 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:40.995252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:40.995342 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:40.995776 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:40.995778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:40.995952 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:40.996061 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.996068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.996846 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.996852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.997432 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:40.997440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.997890 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.997893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:40.998026 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:40.998598 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:40.998609 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:41.000022 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:41.002081 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.002084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.002086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.002349 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.002647 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.002652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.002654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.002740 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.003136 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:41.003139 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:41.003229 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 13:52:41.003351 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 13:52:41.003539 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":21}
I20260926 13:52:41.003898 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:40.995086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.10.236.253:36553)
0926 13:52:40.995505 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:40.995509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:40.995520 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:52:40.995554 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:40.995559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:40.995602 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:40.995668 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:40.996176 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.996760 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.997542 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:40.997789 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:40.997893 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.000162 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.000165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.000166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.000186 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.001932 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.002450 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.002457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.002459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.002541 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.002840 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.002844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.002845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.003032 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.003147 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.003410 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.003654 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":325}
I20260926 13:52:41.004575 24663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:41.005349 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.005039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50304)
0926 13:52:41.005175 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.005281 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 13:52:41.006039 24614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
I20260926 13:52:41.006964 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.006774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50314)
0926 13:52:41.006875 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.006917 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 13:52:41.007133 24663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:35527: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
I20260926 13:52:41.028092 24588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } attempt: 1
I20260926 13:52:41.029062 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.028741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50322)
0926 13:52:41.028916 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.028978 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 13:52:41.029820 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.029600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50324)
0926 13:52:41.029718 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.029769 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 13:52:41.029949 24588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:35527: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
I20260926 13:52:41.056417 24663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } attempt: 1
I20260926 13:52:41.057363 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.057050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50336)
0926 13:52:41.057211 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.057278 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 13:52:41.058106 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.057894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50352)
0926 13:52:41.058003 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.058056 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 13:52:41.058235 24663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:35527: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
I20260926 13:52:41.105149 24588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } attempt: 2
I20260926 13:52:41.106083 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.105738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50366)
0926 13:52:41.105926 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.106000 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260926 13:52:41.106834 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.106608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50380)
0926 13:52:41.106722 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.106778 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:52:41.107132 24588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:35527: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
W20260926 13:52:41.123414 24669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:41.123811 24669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:41.123855 24669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 13:52:41.126431 24663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } attempt: 2
I20260926 13:52:41.127331 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.127011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50390)
0926 13:52:41.127169 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.127240 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 13:52:41.128181 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.127872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50396)
0926 13:52:41.127975 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.128023 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 13:52:41.128213 24669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:41.128266 24669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:41.128306 24669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:41.128347 24669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:41.128381 24669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:52:41.128381 24663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:35527: Network error: Client connection negotiation failed: client connection to 127.10.236.252:35527: connect: Connection refused (error 111)
I20260926 13:52:41.134019 24669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:36339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:38517,127.10.236.253:36553,127.10.236.252:35527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab
--principal=kudu/127.10.236.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:35527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.236.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:41.135224 24669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:41.136449 24669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:41.144961 24678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:41.145021 24676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:41.144980 24675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:41.145293 24669 server_base.cc:1034] running on GCE node
I20260926 13:52:41.146133 24669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:41.147392 24669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:41.148608 24669 hybrid_clock.cc:648] HybridClock initialized: now 1790430761148590 us; error 31 us; skew 500 ppm
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430761, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:41.151696 24669 init.cc:377] Logged in from keytab as kudu/127.10.236.252@KRBTEST.COM (short username kudu)
I20260926 13:52:41.152649 24669 webserver.cc:460] Webserver started at http://127.10.236.252:35783/ using document root <none> and password file <none>
I20260926 13:52:41.153283 24669 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:41.153379 24669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:41.153615 24669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:41.155808 24669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data/instance:
uuid: "771e77cd8b344f929baf3039b8af0372"
format_stamp: "Formatted at 2026-09-26 13:52:41 on dist-test-slave-r34j"
server_key: "48e270453fd532369a8ecda3b502be4b"
server_key_iv: "0f32bcc0238844f8cdfa0129e082d1ac"
server_key_version: "encryptionkey@0"
I20260926 13:52:41.156399 24669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal/instance:
uuid: "771e77cd8b344f929baf3039b8af0372"
format_stamp: "Formatted at 2026-09-26 13:52:41 on dist-test-slave-r34j"
server_key: "48e270453fd532369a8ecda3b502be4b"
server_key_iv: "0f32bcc0238844f8cdfa0129e082d1ac"
server_key_version: "encryptionkey@0"
I20260926 13:52:41.160223 24669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:52:41.162817 24685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:41.164243 24669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:52:41.164394 24669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal
uuid: "771e77cd8b344f929baf3039b8af0372"
format_stamp: "Formatted at 2026-09-26 13:52:41 on dist-test-slave-r34j"
server_key: "48e270453fd532369a8ecda3b502be4b"
server_key_iv: "0f32bcc0238844f8cdfa0129e082d1ac"
server_key_version: "encryptionkey@0"
I20260926 13:52:41.164515 24669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:41.180315 24669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:41.183406 24669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:41.183661 24669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:41.190562 24669 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.252:35527
I20260926 13:52:41.190865 24737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.252:35527 every 8 connection(s)
I20260926 13:52:41.191656 24669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
I20260926 13:52:41.194468 24738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:41.198453 24738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:41.201970 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24669
I20260926 13:52:41.202114 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790430594225530-11187-0/minicluster-data/master-2/wal/instance
I20260926 13:52:41.202401 11187 external_mini_cluster.cc:1428] Setting key 62c85a6f15ff181cb0a4e7899f289461
I20260926 13:52:41.206306 24588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } attempt: 3
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430761, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:52:41.215827 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.200496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.10.236.254:38517)
0926 13:52:41.200670 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.200674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.201568 (+ 894us) server_negotiation.cc:407] Connection header received
0926 13:52:41.202839 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.202842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.202901 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.203004 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.204688 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.205488 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.207092 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.207443 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.207567 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.211191 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.211195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.211197 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.211224 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.213197 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.213836 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.213840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.213843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.213922 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.214340 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.214345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.214347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.214590 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.214689 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.215529 (+ 840us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.215646 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 13:52:41.216216 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.206740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50408)
0926 13:52:41.206875 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.206889 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:41.207004 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:41.207451 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:41.207453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:41.207674 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:41.207763 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.207782 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.208883 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.208889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.209480 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.209491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.209952 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.209957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.210061 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.210465 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:41.210480 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:41.211918 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:41.214666 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.214669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.214671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.214912 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.215234 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.215238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.215243 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.215335 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.215776 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:41.215780 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:41.215871 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 13:52:41.215951 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 13:52:41.216068 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 13:52:41.216775 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.200913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:38517 (local address 127.0.0.1:58172)
0926 13:52:41.201407 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.201450 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:41.202605 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:41.203622 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:41.203629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:41.204059 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:41.204507 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.204527 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.205627 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.205630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.206992 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.207004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.207613 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.207618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.207795 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.208477 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:41.208520 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:41.211056 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:41.213310 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.213320 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.213330 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.213702 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.214089 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.214092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.214094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.214239 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.214684 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:41.214690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:41.215005 (+ 315us) client_negotiation.cc:769] Sending connection context
0926 13:52:41.215234 (+ 229us) client_negotiation.cc:240] Negotiation successful
0926 13:52:41.215600 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":34,"thread_start_us":118,"threads_started":1}
I20260926 13:52:41.216775 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.203056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.10.236.252:35527)
0926 13:52:41.203426 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.203435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.203456 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:52:41.203586 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.203590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.203758 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.203896 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.204722 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.205909 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.207045 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.207716 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.207819 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.211069 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.211076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.211092 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.211120 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.213615 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.214116 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.214124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.214127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.214251 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.214606 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.214613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.214618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.214971 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.215170 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.215610 (+ 440us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.215741 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":290,"thread_start_us":206,"threads_started":1}
I20260926 13:52:41.216919 24743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.206786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50408 (local address 127.10.236.252:35527)
0926 13:52:41.207233 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.207241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.207252 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:52:41.207283 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.207285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.207325 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.207881 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.208007 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.208773 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.209608 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.209835 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.209912 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.212074 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.212081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.212085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.212114 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.214554 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.215034 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.215036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.215038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.215114 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.215429 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.215432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.215434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.215653 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.216521 (+ 868us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.216594 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.216699 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":384,"thread_start_us":113,"threads_started":1}
I20260926 13:52:41.218075 24738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:41.219532 24663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } attempt: 3
I20260926 13:52:41.221522 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: Bootstrap starting.
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430761, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:52:41.224519 24588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: Neither blocks nor log segments found. Creating new log.
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:52:41.225544 24588 log.cc:824] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:41.228045 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: No bootstrap required, opened a new log
I20260926 13:52:41.229631 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.219896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50414)
0926 13:52:41.220139 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.220156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:41.220237 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:41.220744 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:41.220747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:41.220924 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:41.221008 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.221014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.222117 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.222120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.222932 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.222942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.223586 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.223594 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.223705 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.224092 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:41.224108 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:41.225581 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:41.227782 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.227790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.227794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.228056 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.228400 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.228404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.228408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.228515 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.229016 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:41.229019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:41.229125 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 13:52:41.229279 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 13:52:41.229436 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260926 13:52:41.230309 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.220428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.10.236.252:35527)
0926 13:52:41.220546 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.220550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.220561 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:52:41.220592 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.220594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.220630 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.221107 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.221178 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.222001 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.223076 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.223416 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.224009 (+ 593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.225799 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.225802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.225804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.225824 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.227640 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.228189 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.228194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.228196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.228279 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.228640 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.228646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.228651 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.228897 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.229955 (+ 1058us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.230031 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.230153 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:52:41.231190 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.218801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.10.236.253:36553)
0926 13:52:41.218972 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.218979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.218996 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:52:41.219099 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.219102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.219154 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.219242 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.219726 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.220458 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.221528 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.221837 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.221912 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.227113 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.227117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.227119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.227144 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.228849 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.229340 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.229343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.229345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.229426 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.230080 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.230084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.230085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.230305 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.230380 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.230931 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.231045 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 13:52:41.231788 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.219385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.10.236.254:38517)
0926 13:52:41.219531 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.219535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.219548 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:52:41.219582 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.219584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.219623 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.219691 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.220546 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.221252 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.222625 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.222963 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.223040 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.227079 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.227085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.227090 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.227118 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.229278 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.229763 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.229770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.229774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.229865 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.230175 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.230178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.230180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.230397 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.230548 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.230912 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.231649 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:52:41.231819 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.218594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:36553 (local address 127.0.0.1:45850)
0926 13:52:41.218929 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.218943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:41.219015 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:41.219277 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:41.219280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:41.219454 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:41.219592 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.219599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.220600 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.220603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.221387 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.221397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.222058 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.222061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.222155 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.222648 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:41.222662 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:41.226425 (+ 3763us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x33fc86b8. stack: 00007f96578fbc78 00007f96578fc1a9 00007f96574be1d7 00000000005984a6 00000000005984f9 000000000059c488 00007f96578f82ab 00007f96578f7ad0 00007f965bfd00ee 00007f965bf52cae 00007f965bf5207b 00007f965bf4ee17 00007f965bf8f832 00007f965bf9068c 00007f965bfa60d3 00007f965bfa954b
0926 13:52:41.226485 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:41.228972 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.228975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.228977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.229211 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.229533 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.229535 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.229537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.229623 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.230656 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:41.230659 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:41.230749 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 13:52:41.231518 (+ 769us) client_negotiation.cc:240] Negotiation successful
0926 13:52:41.231653 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"spinlock_wait_cycles":4081536}
I20260926 13:52:41.232656 24588 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.232838 24738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:52:41.232903 24588 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:41.232949 24588 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc8a579169db489fbfccf7281b96622c, State: Initialized, Role: FOLLOWER
I20260926 13:52:41.233459 24663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: Bootstrap starting.
I20260926 13:52:41.233413 24588 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.233886 24746 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.234011 24746 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:52:41.234898 24588 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:41.236316 24663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:41.237205 24663 log.cc:824] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:41.239784 24663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: No bootstrap required, opened a new log
W20260926 13:52:41.241348 24757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:52:41.241406 24757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:52:41.239825 24520 master_runner.cc:186] Error getting master registration for 127.10.236.253:36553: OK, instance_id { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" instance_seqno: 1790430760961452 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:52:41.243714 24520 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
Sep 26 13:52:41 dist-test-slave-r34j krb5kdc[24507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430761, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:52:41.245471 24663 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.245713 24663 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:41.245783 24663 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e885f840744b54a349c69c5ce92264, State: Initialized, Role: FOLLOWER
I20260926 13:52:41.246345 24663 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.247833 24663 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:41.248986 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.233255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:35527 (local address 127.0.0.1:50430)
0926 13:52:41.234399 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:41.234411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:41.234484 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:41.234747 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:41.234750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:41.234915 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:41.235055 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.235062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.236126 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.236131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.237076 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:41.237086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.237664 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.237669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:41.239063 (+ 1394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.239567 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:41.239582 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:41.245096 (+ 5514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:41.247299 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.247302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.247304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.247581 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.247868 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:41.247870 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:41.247872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:41.248004 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:41.248432 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:41.248437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:41.248525 (+ 88us) client_negotiation.cc:769] Sending connection context
0926 13:52:41.248621 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 13:52:41.248780 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260926 13:52:41.249722 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.233722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45864 (local address 127.10.236.253:36553)
0926 13:52:41.233935 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.233939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.233952 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:52:41.233985 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.233988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.234025 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.234105 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.234843 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.235607 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.236691 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.237039 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.237139 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.244752 (+ 7613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.244756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.244759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.244793 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.247108 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.247694 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.247698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.247699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.247773 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.248082 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.248086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.248088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.248314 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.248488 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.249450 (+ 962us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.249578 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260926 13:52:41.249730 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.233584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.10.236.252:35527)
0926 13:52:41.234317 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.234321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.234552 (+ 231us) server_negotiation.cc:407] Connection header received
0926 13:52:41.234576 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.234579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.234609 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.234664 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.235164 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.236012 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.237206 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.237564 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.237663 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.245252 (+ 7589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.245256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.245258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.245285 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.247194 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.247684 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.247687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.247689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.247773 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.248096 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.248098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.248100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.248308 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.248398 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.249505 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.249605 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":652}
I20260926 13:52:41.253376 24738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: Bootstrap starting.
I20260926 13:52:41.253691 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:52:41.256278 24738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:41.257169 24738 log.cc:824] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: Log is configured to *not* fsync() on all Append() calls
W20260926 13:52:41.254467 24771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:52:41.258220 24771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:52:41.254765 24760 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.258472 24760 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:52:41.263734 24595 master_runner.cc:186] Error getting master registration for 127.10.236.252:35527: OK, instance_id { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" instance_seqno: 1790430761188731 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:52:41.267995 24595 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:52:41.271631 24738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: No bootstrap required, opened a new log
I20260926 13:52:41.272315 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.257042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.10.236.254:38517)
0926 13:52:41.257306 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.257311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.258267 (+ 956us) server_negotiation.cc:407] Connection header received
0926 13:52:41.258484 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.258488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.258539 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.258593 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.259638 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.260439 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.262986 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.263348 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.263424 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.267852 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.267856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.267859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.267890 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.269640 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.270135 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.270139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.270141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.270239 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.271326 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.271330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.271332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.271585 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.271685 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.272020 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.272143 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260926 13:52:41.276270 24738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.276628 24738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:41.276715 24738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 771e77cd8b344f929baf3039b8af0372, State: Initialized, Role: FOLLOWER
I20260926 13:52:41.277300 24738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.277433 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.257177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45868 (local address 127.10.236.253:36553)
0926 13:52:41.257389 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.257393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.261786 (+ 4393us) server_negotiation.cc:407] Connection header received
0926 13:52:41.261948 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.261952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.262002 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.262051 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.263970 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.264744 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.266938 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.267292 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.267361 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.273820 (+ 6459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.273824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.273828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.273859 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.275529 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.276013 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.276017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.276019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.276100 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.276412 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.276416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.276418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.276632 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.276744 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.277075 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.277265 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260926 13:52:41.278916 24738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:41.279726 24781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.279922 24781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 13:52:41.284014 24792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:52:41.284163 24792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:52:41.289613 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:41.257632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.10.236.252:35527)
0926 13:52:41.257749 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:41.257754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:41.269037 (+ 11283us) server_negotiation.cc:407] Connection header received
0926 13:52:41.269241 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:41.269246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:41.269299 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:41.269354 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:41.281390 (+ 12036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.282168 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.283308 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:41.283664 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:41.283740 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:41.285930 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:41.285934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:41.285937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:41.285964 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:41.287835 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.288303 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.288307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.288309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.288391 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:41.288669 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:41.288673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:41.288675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:41.288885 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:41.288981 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:41.289335 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 13:52:41.289456 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:52:41.290338 24669 master_runner.cc:263] No leader master found from master 771e77cd8b344f929baf3039b8af0372
I20260926 13:52:41.290444 24669 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 13:52:41.435235 24691 tablet.cc:2369] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:52:41.457286 24616 tablet.cc:2369] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:52:41.481570 24541 tablet.cc:2369] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:52:41.500351 24781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:41.500573 24781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.501645 24781 leader_election.cc:288] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc8a579169db489fbfccf7281b96622c (127.10.236.254:38517), d3e885f840744b54a349c69c5ce92264 (127.10.236.253:36553)
I20260926 13:52:41.502243 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "771e77cd8b344f929baf3039b8af0372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc8a579169db489fbfccf7281b96622c" is_pre_election: true
I20260926 13:52:41.502243 24636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "771e77cd8b344f929baf3039b8af0372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e885f840744b54a349c69c5ce92264" is_pre_election: true
I20260926 13:52:41.502633 24636 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 771e77cd8b344f929baf3039b8af0372 in term 0.
I20260926 13:52:41.502606 24562 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 771e77cd8b344f929baf3039b8af0372 in term 0.
I20260926 13:52:41.503060 24687 leader_election.cc:302] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 771e77cd8b344f929baf3039b8af0372, d3e885f840744b54a349c69c5ce92264; no voters:
I20260926 13:52:41.503412 24781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:52:41.503523 24781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:52:41.503571 24781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:41.504623 24781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.504962 24781 leader_election.cc:288] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [CANDIDATE]: Term 1 election: Requested vote from peers cc8a579169db489fbfccf7281b96622c (127.10.236.254:38517), d3e885f840744b54a349c69c5ce92264 (127.10.236.253:36553)
I20260926 13:52:41.505394 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "771e77cd8b344f929baf3039b8af0372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc8a579169db489fbfccf7281b96622c"
I20260926 13:52:41.505394 24636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "771e77cd8b344f929baf3039b8af0372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e885f840744b54a349c69c5ce92264"
I20260926 13:52:41.505553 24636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:41.505563 24562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:41.506910 24562 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 771e77cd8b344f929baf3039b8af0372 in term 1.
I20260926 13:52:41.506910 24636 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 771e77cd8b344f929baf3039b8af0372 in term 1.
I20260926 13:52:41.507289 24687 leader_election.cc:302] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 771e77cd8b344f929baf3039b8af0372, cc8a579169db489fbfccf7281b96622c; no voters:
I20260926 13:52:41.507504 24781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:41.507757 24781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [term 1 LEADER]: Becoming Leader. State: Replica: 771e77cd8b344f929baf3039b8af0372, State: Running, Role: LEADER
I20260926 13:52:41.508108 24781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } }
I20260926 13:52:41.509893 24794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 771e77cd8b344f929baf3039b8af0372. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.510031 24794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:41.510435 24796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:41.511301 24796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:41.516269 24636 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [term 1 FOLLOWER]: Refusing update from remote peer 771e77cd8b344f929baf3039b8af0372: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:41.516285 24562 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [term 1 FOLLOWER]: Refusing update from remote peer 771e77cd8b344f929baf3039b8af0372: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:41.517021 24794 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:41.517184 24781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:41.521147 24760 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 771e77cd8b344f929baf3039b8af0372. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.521147 24746 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 771e77cd8b344f929baf3039b8af0372. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.521287 24746 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:52:41.521287 24760 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:52:41.522774 24797 mvcc.cc:205] Tried to move back new op lower bound from 7333604399163596800 to 7333604399139819520. Current Snapshot: MvccSnapshot[applied={T|T < 7333604399163596800}]
I20260926 13:52:41.523365 24794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.523514 24794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:41.524102 24760 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.524210 24760 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:52:41.524389 24781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.524507 24781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:41.524652 24798 mvcc.cc:205] Tried to move back new op lower bound from 7333604399163596800 to 7333604399139819520. Current Snapshot: MvccSnapshot[applied={T|T < 7333604399163596800}]
I20260926 13:52:41.525524 24746 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "771e77cd8b344f929baf3039b8af0372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc8a579169db489fbfccf7281b96622c" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 38517 } } peers { permanent_uuid: "d3e885f840744b54a349c69c5ce92264" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 36553 } } peers { permanent_uuid: "771e77cd8b344f929baf3039b8af0372" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 35527 } } }
I20260926 13:52:41.525641 24746 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:52:41.526118 24799 mvcc.cc:205] Tried to move back new op lower bound from 7333604399163596800 to 7333604399139819520. Current Snapshot: MvccSnapshot[applied={T|T < 7333604399163596800}]
I20260926 13:52:41.526280 24796 catalog_manager.cc:1347] Generated new cluster ID: 9953eca60c1a4e93ab98616b592203d7
I20260926 13:52:41.526345 24796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:52:41.552814 24796 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:52:41.553957 24796 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:52:41.565958 24796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 771e77cd8b344f929baf3039b8af0372: Generated new TSK 0
I20260926 13:52:41.566953 24796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:52:42.242749 24757 catalog_manager.cc:1259] Loaded cluster ID: 9953eca60c1a4e93ab98616b592203d7
I20260926 13:52:42.242846 24757 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: loading cluster ID for follower catalog manager: success
I20260926 13:52:42.244663 24757 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: acquiring CA information for follower catalog manager: success
I20260926 13:52:42.245993 24757 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc8a579169db489fbfccf7281b96622c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:52:42.259678 24771 catalog_manager.cc:1259] Loaded cluster ID: 9953eca60c1a4e93ab98616b592203d7
I20260926 13:52:42.259760 24771 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: loading cluster ID for follower catalog manager: success
I20260926 13:52:42.261406 24771 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: acquiring CA information for follower catalog manager: success
I20260926 13:52:42.262580 24771 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3e885f840744b54a349c69c5ce92264: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:52:42.644714 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24520
I20260926 13:52:42.651584 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24595
I20260926 13:52:42.659204 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24669
2026-09-26T13:52:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3337 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:52:42 dist-test-slave-r34j krb5kdc[24811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:52:42 dist-test-slave-r34j krb5kdc[24811](info): set up 2 sockets
Sep 26 13:52:42 dist-test-slave-r34j krb5kdc[24811](info): commencing operation
krb5kdc: starting...
W20260926 13:52:43.732203 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:52:43 dist-test-slave-r34j krb5kdc[24811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:52:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:52:43.864075 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45799
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:45799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:43.980515 24824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:43.980866 24824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:43.980943 24824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:43.985507 24824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:43.985633 24824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:43.985678 24824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:43.985718 24824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:43.985749 24824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:43.991379 24824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:45799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:45799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:43.992605 24824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:43.993698 24824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:44.003114 24829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:44.003144 24830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:44.003114 24832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:44.003716 24824 server_base.cc:1034] running on GCE node
I20260926 13:52:44.004421 24824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:44.005674 24824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:44.006892 24824 hybrid_clock.cc:648] HybridClock initialized: now 1790430764006855 us; error 48 us; skew 500 ppm
Sep 26 13:52:44 dist-test-slave-r34j krb5kdc[24811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430764, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:44.009954 24824 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:44.010985 24824 webserver.cc:460] Webserver started at http://127.10.236.254:44283/ using document root <none> and password file <none>
I20260926 13:52:44.011643 24824 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:44.011714 24824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:44.011904 24824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:44.014029 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "9cb849dee65b41aca4ab637aa62d2332"
format_stamp: "Formatted at 2026-09-26 13:52:44 on dist-test-slave-r34j"
server_key: "6cf73b7a0083dff719e6f480a15e0f86"
server_key_iv: "ff73afbc0c7f766908ea7e0804559509"
server_key_version: "encryptionkey@0"
I20260926 13:52:44.014557 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "9cb849dee65b41aca4ab637aa62d2332"
format_stamp: "Formatted at 2026-09-26 13:52:44 on dist-test-slave-r34j"
server_key: "6cf73b7a0083dff719e6f480a15e0f86"
server_key_iv: "ff73afbc0c7f766908ea7e0804559509"
server_key_version: "encryptionkey@0"
I20260926 13:52:44.018517 24824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260926 13:52:44.021718 24839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:44.023152 24824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:52:44.023309 24824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "9cb849dee65b41aca4ab637aa62d2332"
format_stamp: "Formatted at 2026-09-26 13:52:44 on dist-test-slave-r34j"
server_key: "6cf73b7a0083dff719e6f480a15e0f86"
server_key_iv: "ff73afbc0c7f766908ea7e0804559509"
server_key_version: "encryptionkey@0"
I20260926 13:52:44.023423 24824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:44.049189 24824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:44.052495 24824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:44.052662 24824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:44.060313 24824 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:45799
I20260926 13:52:44.060335 24891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:45799 every 8 connection(s)
I20260926 13:52:44.061312 24824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:44.061619 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24824
I20260926 13:52:44.061764 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:52:44.062095 11187 external_mini_cluster.cc:1428] Setting key 46dd11502aa9f5dd33ccdeaa8b7425ac
I20260926 13:52:44.064600 24892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:44.070658 24892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332: Bootstrap starting.
Sep 26 13:52:44 dist-test-slave-r34j krb5kdc[24811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:44.073405 24892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:44.074186 24892 log.cc:824] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:44.076505 24892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332: No bootstrap required, opened a new log
I20260926 13:52:44.077333 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:44.063582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52346 (local address 127.10.236.254:45799)
0926 13:52:44.064020 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:44.064035 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:44.064070 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:52:44.064786 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:44.064807 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:44.065141 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:44.065488 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:44.066374 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:44.067394 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:44.068309 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:44.068778 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:44.068946 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:44.071914 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:44.071919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:44.071931 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:44.071962 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:44.073958 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:44.074445 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:44.074451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:44.074456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:44.074592 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:44.075059 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:44.075067 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:44.075068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:44.075550 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:44.075707 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:44.076073 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 13:52:44.076460 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":143,"threads_started":1}
I20260926 13:52:44.080477 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:52:44.080276 24892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb849dee65b41aca4ab637aa62d2332" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45799 } }
I20260926 13:52:44.080592 24892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:44.080681 24892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb849dee65b41aca4ab637aa62d2332, State: Initialized, Role: FOLLOWER
I20260926 13:52:44.081118 24892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb849dee65b41aca4ab637aa62d2332" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45799 } }
I20260926 13:52:44.081272 24892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:52:44.081362 24892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:52:44.081468 24892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:44.082589 24892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb849dee65b41aca4ab637aa62d2332" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45799 } }
I20260926 13:52:44.082904 24892 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb849dee65b41aca4ab637aa62d2332; no voters:
I20260926 13:52:44.083472 24892 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:52:44.083727 24897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:44.084013 24897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb849dee65b41aca4ab637aa62d2332, State: Running, Role: LEADER
I20260926 13:52:44.084384 24897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb849dee65b41aca4ab637aa62d2332" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45799 } }
I20260926 13:52:44.084918 24892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:44.086076 24898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb849dee65b41aca4ab637aa62d2332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb849dee65b41aca4ab637aa62d2332" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45799 } } }
I20260926 13:52:44.086229 24898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:44.086611 24899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb849dee65b41aca4ab637aa62d2332. Latest consensus state: current_term: 1 leader_uuid: "9cb849dee65b41aca4ab637aa62d2332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb849dee65b41aca4ab637aa62d2332" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 45799 } } }
I20260926 13:52:44.086694 24906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:44.086792 24899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332 [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:44.089764 24906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:44.095517 24906 catalog_manager.cc:1347] Generated new cluster ID: fd238f5fba1249409ef342a15ba40242
I20260926 13:52:44.095605 24906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:52:44.123919 24906 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:52:44.125267 24906 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:52:44.144665 24906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb849dee65b41aca4ab637aa62d2332: Generated new TSK 0
I20260926 13:52:44.145514 24906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:52:44.188267 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24824
2026-09-26T13:52:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1525 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:52:44 dist-test-slave-r34j krb5kdc[24919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:52:44 dist-test-slave-r34j krb5kdc[24919](info): set up 2 sockets
Sep 26 13:52:44 dist-test-slave-r34j krb5kdc[24919](info): commencing operation
krb5kdc: starting...
W20260926 13:52:45.239843 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:52:45 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:52:45.367372 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43247
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:43247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:45.478785 24932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:45.479113 24932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:45.479187 24932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:45.483258 24932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:45.483371 24932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:45.483410 24932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:45.483448 24932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:45.483502 24932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:45.489574 24932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:43247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:45.490700 24932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:45.491793 24932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:45.500478 24937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:45.500526 24938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:45.500531 24940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:45.500607 24932 server_base.cc:1034] running on GCE node
I20260926 13:52:45.501365 24932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:45.502437 24932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:45.503583 24932 hybrid_clock.cc:648] HybridClock initialized: now 1790430765503566 us; error 28 us; skew 500 ppm
Sep 26 13:52:45 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430765, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:45.511430 24932 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:45.512461 24932 webserver.cc:460] Webserver started at http://127.10.236.254:46051/ using document root <none> and password file <none>
I20260926 13:52:45.513037 24932 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:45.513098 24932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:45.513319 24932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:45.515340 24932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "6c57251a0e1e45c0a16d51336c79eaad"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "c4fadb97ac4c97fdf64d71fc823258bb"
server_key_iv: "6566f5145c7a9d47173a4a3c427b41a8"
server_key_version: "encryptionkey@0"
I20260926 13:52:45.515965 24932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "6c57251a0e1e45c0a16d51336c79eaad"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "c4fadb97ac4c97fdf64d71fc823258bb"
server_key_iv: "6566f5145c7a9d47173a4a3c427b41a8"
server_key_version: "encryptionkey@0"
I20260926 13:52:45.519742 24932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:52:45.522156 24947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:45.523653 24932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:45.523785 24932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "6c57251a0e1e45c0a16d51336c79eaad"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "c4fadb97ac4c97fdf64d71fc823258bb"
server_key_iv: "6566f5145c7a9d47173a4a3c427b41a8"
server_key_version: "encryptionkey@0"
I20260926 13:52:45.523906 24932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:45.534953 24932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:45.537940 24932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:45.538142 24932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:45.545287 24932 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:43247
I20260926 13:52:45.545595 24999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:43247 every 8 connection(s)
I20260926 13:52:45.546303 24932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:45.549129 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:45.554209 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 24932
I20260926 13:52:45.554395 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:52:45.554709 11187 external_mini_cluster.cc:1428] Setting key eed0f1bd8666bdd7dc675bd6a8187291
I20260926 13:52:45.554715 25000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap starting.
I20260926 13:52:45.557164 25000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:45.557899 25000 log.cc:824] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:45.560128 25000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: No bootstrap required, opened a new log
Sep 26 13:52:45 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:45.564786 25000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:45.565043 25000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:45.565104 25000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c57251a0e1e45c0a16d51336c79eaad, State: Initialized, Role: FOLLOWER
I20260926 13:52:45.565600 25000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:45.565774 25000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:52:45.565903 25000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:52:45.566038 25000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:45.567045 25000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:45.567411 25000 leader_election.cc:302] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c57251a0e1e45c0a16d51336c79eaad; no voters:
I20260926 13:52:45.567785 25000 leader_election.cc:288] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:52:45.567854 25005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:45.568059 25005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 LEADER]: Becoming Leader. State: Replica: 6c57251a0e1e45c0a16d51336c79eaad, State: Running, Role: LEADER
I20260926 13:52:45.568435 25005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:45.569124 25000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:45.570385 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:45.556207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.10.236.254:43247)
0926 13:52:45.556666 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:45.556684 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:45.556722 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:52:45.557492 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:45.557511 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:45.557874 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:45.558224 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:45.559028 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:45.560368 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:45.561533 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:45.562069 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:45.562222 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:45.565096 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:45.565102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:45.565116 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:45.565171 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:45.567059 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:45.567576 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:45.567582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:45.567589 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:45.567713 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:45.568031 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:45.568034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:45.568036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:45.568576 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:45.568797 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:45.569113 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 13:52:45.569429 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":157,"threads_started":1}
I20260926 13:52:45.570350 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c57251a0e1e45c0a16d51336c79eaad. Latest consensus state: current_term: 1 leader_uuid: "6c57251a0e1e45c0a16d51336c79eaad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } } }
I20260926 13:52:45.570679 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:45.570556 25006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c57251a0e1e45c0a16d51336c79eaad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } } }
I20260926 13:52:45.570924 25006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:45.571676 25015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:45.574170 25015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:45.579679 25015 catalog_manager.cc:1347] Generated new cluster ID: ac4f58942b8743ad9697838ad9210a33
I20260926 13:52:45.579747 25015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:52:45.594180 25015 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:52:45.595451 25015 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:52:45.606854 25015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Generated new TSK 0
I20260926 13:52:45.607869 25015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:52:45.639156 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43247
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:45.752931 25026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:45.753214 25026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:45.753258 25026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:45.757977 25026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:45.758049 25026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:45.758111 25026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:52:45.763751 25026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:45.764919 25026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:45.765957 25026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:45.776170 25034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:45.776170 25032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:45.776171 25031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:45.776448 25026 server_base.cc:1034] running on GCE node
I20260926 13:52:45.776993 25026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:45.777716 25026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:45.778935 25026 hybrid_clock.cc:648] HybridClock initialized: now 1790430765778895 us; error 53 us; skew 500 ppm
Sep 26 13:52:45 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430765, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:45.782061 25026 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:52:45.783015 25026 webserver.cc:460] Webserver started at http://127.10.236.193:34717/ using document root <none> and password file <none>
I20260926 13:52:45.783625 25026 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:45.783695 25026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:45.783872 25026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:45.786033 25026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "a50c6ac8632b462b8a04a35471fd0c4d"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "d0ebbbedf2feaa3ed99b1522c38f1b97"
server_key_iv: "d672dc2ea463e4cb152bb142d86f3e89"
server_key_version: "encryptionkey@0"
I20260926 13:52:45.786564 25026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "a50c6ac8632b462b8a04a35471fd0c4d"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "d0ebbbedf2feaa3ed99b1522c38f1b97"
server_key_iv: "d672dc2ea463e4cb152bb142d86f3e89"
server_key_version: "encryptionkey@0"
I20260926 13:52:45.790508 25026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260926 13:52:45.793061 25041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:45.794428 25026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:52:45.794559 25026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "a50c6ac8632b462b8a04a35471fd0c4d"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "d0ebbbedf2feaa3ed99b1522c38f1b97"
server_key_iv: "d672dc2ea463e4cb152bb142d86f3e89"
server_key_version: "encryptionkey@0"
I20260926 13:52:45.794688 25026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:45.816403 25026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:45.819905 25026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:45.820119 25026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:45.820731 25026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:45.821625 25026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:45.821672 25026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:45.821715 25026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:45.821731 25026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:45.831161 25026 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:38219
I20260926 13:52:45.831180 25154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:38219 every 8 connection(s)
I20260926 13:52:45.832173 25026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:52:45.836079 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25026
I20260926 13:52:45.836225 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:52:45.836534 11187 external_mini_cluster.cc:1428] Setting key fac191c7d8d48014f3b13f08e9a531bd
Sep 26 13:52:45 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430765, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:52:45.847121 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:45.834124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:59671 (local address 127.10.236.254:43247)
0926 13:52:45.834279 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:45.834289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:45.835051 (+ 762us) server_negotiation.cc:407] Connection header received
0926 13:52:45.836873 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:45.836877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:45.836937 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:45.837048 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:45.838416 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:45.838938 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:45.840165 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:45.840410 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:45.840525 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:45.843141 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:45.843144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:45.843146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:45.843171 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:45.844646 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:45.845316 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:45.845322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:45.845324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:45.845448 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:45.845934 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:45.845938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:45.845940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:45.846228 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:45.846338 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:45.846839 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 13:52:45.846951 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 13:52:45.848057 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:45.834431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:59671)
0926 13:52:45.834899 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:45.834942 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:45.836642 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:45.837228 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:45.837258 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:45.837753 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:45.838270 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:45.838285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:45.839066 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:45.839069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:45.840038 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:45.840048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:45.840567 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:45.840571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:45.840741 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:45.841424 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:45.841441 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:45.842951 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:45.844769 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:45.844775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:45.844791 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:45.845165 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:45.845580 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:45.845584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:45.845588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:45.845811 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:45.846356 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:45.846361 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:45.846608 (+ 247us) client_negotiation.cc:769] Sending connection context
0926 13:52:45.846780 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 13:52:45.847110 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:52:45.849483 25155 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43247
I20260926 13:52:45.849771 25155 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:45.850373 25155 heartbeater.cc:511] Master 127.10.236.254:43247 requested a full tablet report, sending...
I20260926 13:52:45.852268 24964 ts_manager.cc:194] Registered new tserver with Master: a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219)
I20260926 13:52:45.854034 24964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:59671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:52:45.867118 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43247
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:45.979519 25160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:45.979823 25160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:45.979871 25160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:45.984324 25160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:45.984396 25160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:45.984460 25160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:52:45.989956 25160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:45.991004 25160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:45.992080 25160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:46.002522 25168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.002630 25160 server_base.cc:1034] running on GCE node
W20260926 13:52:46.002614 25166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:46.002522 25165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.003166 25160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:46.003942 25160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:46.005136 25160 hybrid_clock.cc:648] HybridClock initialized: now 1790430766005112 us; error 30 us; skew 500 ppm
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:46.008008 25160 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:52:46.008876 25160 webserver.cc:460] Webserver started at http://127.10.236.194:40749/ using document root <none> and password file <none>
I20260926 13:52:46.009415 25160 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:46.009474 25160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:46.009639 25160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:46.011770 25160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "7024a7263cd3483eb765ae7d65cb2ce4"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "41572d4ee005d8b2addeade9d0c00e64"
server_key_iv: "2bd7161b0a0c19881ff4fed6e5010ea9"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.012336 25160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "7024a7263cd3483eb765ae7d65cb2ce4"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "41572d4ee005d8b2addeade9d0c00e64"
server_key_iv: "2bd7161b0a0c19881ff4fed6e5010ea9"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.016316 25160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:52:46.018796 25175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.020195 25160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:52:46.020345 25160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "7024a7263cd3483eb765ae7d65cb2ce4"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "41572d4ee005d8b2addeade9d0c00e64"
server_key_iv: "2bd7161b0a0c19881ff4fed6e5010ea9"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.020462 25160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:46.063637 25160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:46.066917 25160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:46.067129 25160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:46.067794 25160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:46.068648 25160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:46.068696 25160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.068734 25160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:46.068749 25160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.078193 25160 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33569
I20260926 13:52:46.078183 25288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33569 every 8 connection(s)
I20260926 13:52:46.079124 25160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:52:46.084460 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25160
I20260926 13:52:46.084563 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:52:46.084908 11187 external_mini_cluster.cc:1428] Setting key 6b7d0764ca2ff29887f487c3faea244e
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:52:46.094940 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.081000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:60679 (local address 127.10.236.254:43247)
0926 13:52:46.081165 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.081170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.082042 (+ 872us) server_negotiation.cc:407] Connection header received
0926 13:52:46.083354 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.083358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.083405 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.083525 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:46.084791 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.085474 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.086754 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.086993 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.087106 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.089917 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:46.089921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:46.089923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:46.089957 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:46.092743 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.093300 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.093305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.093307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.093397 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.093750 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.093755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.093756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.093954 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:46.094061 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.094606 (+ 545us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.094762 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 13:52:46.095863 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.081425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:60679)
0926 13:52:46.081897 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.081938 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.083131 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.083694 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.083700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.084077 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:46.084601 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.084644 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.085598 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.085601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.086624 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.086634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.087141 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.087146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.087357 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.088056 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:46.088085 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:46.089750 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:46.092875 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:46.092881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:46.092899 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:46.093179 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:46.093492 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:46.093496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:46.093498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:46.093650 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:46.094074 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:46.094083 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:46.094360 (+ 277us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.094564 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.094912 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":118,"threads_started":1}
I20260926 13:52:46.097160 25289 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43247
I20260926 13:52:46.097390 25289 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:52:46.097919 25289 heartbeater.cc:511] Master 127.10.236.254:43247 requested a full tablet report, sending...
I20260926 13:52:46.099066 24964 ts_manager.cc:194] Registered new tserver with Master: 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569)
I20260926 13:52:46.099726 24964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:60679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:52:46.115433 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43247
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:46.228355 25294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:46.228698 25294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:46.228775 25294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:46.232834 25294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:46.232929 25294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:46.233024 25294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:52:46.238224 25294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:46.239372 25294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:46.240425 25294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:46.250272 25299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:46.250208 25300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.250393 25294 server_base.cc:1034] running on GCE node
W20260926 13:52:46.250208 25302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.250963 25294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:46.251739 25294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:46.252929 25294 hybrid_clock.cc:648] HybridClock initialized: now 1790430766252909 us; error 35 us; skew 500 ppm
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:46.255815 25294 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:52:46.256699 25294 webserver.cc:460] Webserver started at http://127.10.236.195:33637/ using document root <none> and password file <none>
I20260926 13:52:46.257272 25294 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:46.257367 25294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:46.257606 25294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:52:46.259682 25294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "b8582728560249a7a84ceddc1aab2e32"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "999926a6a00320b53992389cf427398c"
server_key_iv: "90e8642e614b9c636094116ac84e680a"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.260243 25294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "b8582728560249a7a84ceddc1aab2e32"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "999926a6a00320b53992389cf427398c"
server_key_iv: "90e8642e614b9c636094116ac84e680a"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.264038 25294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:52:46.266476 25309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.267755 25294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:52:46.267896 25294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "b8582728560249a7a84ceddc1aab2e32"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "999926a6a00320b53992389cf427398c"
server_key_iv: "90e8642e614b9c636094116ac84e680a"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.268007 25294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:46.288316 25294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:46.291747 25294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:46.291975 25294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:46.292573 25294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:46.293403 25294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:46.293449 25294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.293489 25294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:46.293504 25294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.302910 25294 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:37033
I20260926 13:52:46.302968 25422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:37033 every 8 connection(s)
I20260926 13:52:46.303923 25294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:52:46.312119 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25294
I20260926 13:52:46.312265 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:52:46.312546 11187 external_mini_cluster.cc:1428] Setting key b3b30c8c8a290a9f13b812b6de0d13a6
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:46.318729 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.305933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:44421 (local address 127.10.236.254:43247)
0926 13:52:46.306136 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.306141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.307053 (+ 912us) server_negotiation.cc:407] Connection header received
0926 13:52:46.308515 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.308517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.308581 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.308665 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:46.309953 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.310512 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.311746 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.311971 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.312079 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.315148 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:46.315151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:46.315153 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:46.315177 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:46.316761 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.317259 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.317262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.317263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.317336 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.317721 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.317724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.317725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.317917 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:46.318006 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.318503 (+ 497us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.318604 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 13:52:46.319593 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.306402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:44421)
0926 13:52:46.306892 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.306936 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.308280 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.308849 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.308856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.309296 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:46.309800 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.309814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.310624 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.310626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.311631 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.311641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.312176 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.312183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.312410 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.313120 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:46.313137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:46.314948 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:46.316865 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:46.316870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:46.316882 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:46.317140 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:46.317459 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:46.317462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:46.317464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:46.317605 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:46.318015 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:46.318020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:46.318253 (+ 233us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.318418 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.318671 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
I20260926 13:52:46.320650 25423 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43247
I20260926 13:52:46.320901 25423 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:46.321388 25423 heartbeater.cc:511] Master 127.10.236.254:43247 requested a full tablet report, sending...
I20260926 13:52:46.322353 24964 ts_manager.cc:194] Registered new tserver with Master: b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
I20260926 13:52:46.322927 24964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:44421
I20260926 13:52:46.326718 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:52:46.336633 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.328894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.10.236.254:43247)
0926 13:52:46.329074 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.329078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.329216 (+ 138us) server_negotiation.cc:407] Connection header received
0926 13:52:46.329377 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.329380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.329432 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.329523 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:46.330267 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.330794 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.331720 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.331952 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.332059 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.333455 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:46.333459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:46.333462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:46.333487 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:46.334840 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.335297 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.335301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.335303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.335385 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.335717 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.335720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.335722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.335951 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:46.336095 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.336345 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.336466 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 13:52:46.339824 24964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:52:46.342854 24964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:52:46.357497 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.351850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37033 (local address 127.0.0.1:34402)
0926 13:52:46.352540 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.352554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.352631 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.353492 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.353497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.353519 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:46.353602 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.353607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.354567 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.354572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.355926 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.355936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.356944 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.356947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.357049 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.357085 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.357220 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.357357 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":80,"threads_started":1}
I20260926 13:52:46.357508 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.352242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.10.236.195:37033)
0926 13:52:46.352900 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.352912 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.352939 (+ 27us) server_negotiation.cc:407] Connection header received
0926 13:52:46.353001 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.353004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.353177 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.353330 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:46.353708 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.354434 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.356058 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.356806 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.356945 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.357084 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.357249 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.357350 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":135,"threads_started":1}
I20260926 13:52:46.357774 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.351447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33569 (local address 127.0.0.1:46450)
0926 13:52:46.352119 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.352170 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.352312 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.353214 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.353223 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.353258 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:46.353414 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.353421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.355100 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.355103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.356347 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.356357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.357367 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.357371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.357461 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.357492 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.357539 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.357606 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":201,"spinlock_wait_cycles":3328,"thread_start_us":96,"threads_started":1}
I20260926 13:52:46.358189 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.352216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.10.236.193:38219)
0926 13:52:46.352725 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.352732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.352756 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:52:46.352832 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.352837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.353048 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.353211 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:46.353581 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.354778 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.356332 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.357139 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.357278 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.357419 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.357924 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.358042 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":73,"threads_started":1}
I20260926 13:52:46.357831 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.351621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:38219 (local address 127.0.0.1:42568)
0926 13:52:46.352482 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.352498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.352592 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.353212 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.353220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.353266 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:46.353450 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.353459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.355051 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.355055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.356216 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.356223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.357469 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.357472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.357580 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.357599 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.357659 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.357714 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":52,"spinlock_wait_cycles":11520,"thread_start_us":120,"threads_started":1}
I20260926 13:52:46.358642 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.351624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.10.236.194:33569)
0926 13:52:46.352837 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.352848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.352867 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:52:46.352929 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.352933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.353071 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.353570 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:46.353736 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.354831 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.356467 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.357243 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.358158 (+ 915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.358334 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.358381 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.358488 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":134,"threads_started":1}
I20260926 13:52:46.361033 25223 tablet_service.cc:1467] Processing CreateTablet for tablet 8c7931987dd14bf08cd960e884320218 (DEFAULT_TABLE table=test-table [id=44596aa849a343f7976695a917d14008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:46.360908 25357 tablet_service.cc:1467] Processing CreateTablet for tablet 8c7931987dd14bf08cd960e884320218 (DEFAULT_TABLE table=test-table [id=44596aa849a343f7976695a917d14008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:46.361392 25089 tablet_service.cc:1467] Processing CreateTablet for tablet 8c7931987dd14bf08cd960e884320218 (DEFAULT_TABLE table=test-table [id=44596aa849a343f7976695a917d14008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:46.362056 25223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c7931987dd14bf08cd960e884320218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:46.362080 25357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c7931987dd14bf08cd960e884320218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:46.362282 25089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c7931987dd14bf08cd960e884320218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:46.368933 25440 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Bootstrap starting.
I20260926 13:52:46.369477 25442 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Bootstrap starting.
I20260926 13:52:46.371822 25441 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap starting.
I20260926 13:52:46.372021 25442 tablet_bootstrap.cc:654] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:46.372021 25440 tablet_bootstrap.cc:654] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:46.372938 25440 log.cc:824] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:46.372938 25442 log.cc:824] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:46.374253 25441 tablet_bootstrap.cc:654] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:46.375041 25442 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: No bootstrap required, opened a new log
I20260926 13:52:46.375073 25441 log.cc:824] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:46.375130 25440 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: No bootstrap required, opened a new log
I20260926 13:52:46.375257 25442 ts_tablet_manager.cc:1397] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:52:46.375298 25440 ts_tablet_manager.cc:1397] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 13:52:46.377105 25441 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: No bootstrap required, opened a new log
I20260926 13:52:46.377308 25441 ts_tablet_manager.cc:1397] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:52:46.378404 25442 raft_consensus.cc:348] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.378772 25442 raft_consensus.cc:374] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:46.378877 25442 raft_consensus.cc:729] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8582728560249a7a84ceddc1aab2e32, State: Initialized, Role: FOLLOWER
I20260926 13:52:46.379199 25440 raft_consensus.cc:348] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.379508 25440 raft_consensus.cc:374] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:46.379595 25440 raft_consensus.cc:729] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7024a7263cd3483eb765ae7d65cb2ce4, State: Initialized, Role: FOLLOWER
I20260926 13:52:46.379427 25442 consensus_queue.cc:260] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.380028 25440 consensus_queue.cc:260] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.380210 25423 heartbeater.cc:503] Master 127.10.236.254:43247 was elected leader, sending a full tablet report...
I20260926 13:52:46.380559 25442 ts_tablet_manager.cc:1428] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260926 13:52:46.380946 25440 ts_tablet_manager.cc:1428] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260926 13:52:46.381443 25289 heartbeater.cc:503] Master 127.10.236.254:43247 was elected leader, sending a full tablet report...
I20260926 13:52:46.382570 25441 raft_consensus.cc:348] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.382826 25441 raft_consensus.cc:374] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:46.382886 25441 raft_consensus.cc:729] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50c6ac8632b462b8a04a35471fd0c4d, State: Initialized, Role: FOLLOWER
I20260926 13:52:46.383294 25441 consensus_queue.cc:260] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.384063 25155 heartbeater.cc:503] Master 127.10.236.254:43247 was elected leader, sending a full tablet report...
I20260926 13:52:46.384361 25441 ts_tablet_manager.cc:1428] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 13:52:46.460184 25448 raft_consensus.cc:482] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:46.460455 25448 raft_consensus.cc:504] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.461855 25448 leader_election.cc:288] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
I20260926 13:52:46.466378 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.462350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:55315 (local address 127.10.236.195:37033)
0926 13:52:46.462499 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.462504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.462828 (+ 324us) server_negotiation.cc:407] Connection header received
0926 13:52:46.462949 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.462952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.463006 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.463103 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:46.463647 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.464163 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.465335 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.465762 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.465880 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.465921 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.466164 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.466257 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:52:46.466403 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.462220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37033 (local address 127.10.236.193:55315)
0926 13:52:46.462659 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.462676 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.462833 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.463146 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.463149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.463178 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:46.463466 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.463495 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.464314 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.464317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.465199 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.465209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.465919 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.465924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.466029 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.466051 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.466134 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.466220 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":126,"spinlock_wait_cycles":28160,"thread_start_us":143,"threads_started":1}
I20260926 13:52:46.466447 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.462220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33569 (local address 127.10.236.193:42901)
0926 13:52:46.462716 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.462729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.462833 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.463146 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.463149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.463177 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:46.463466 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.463474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.464256 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.464264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.465214 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.465225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.466010 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.466013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.466090 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.466136 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.466214 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.466276 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":84,"spinlock_wait_cycles":15488,"thread_start_us":102,"threads_started":1}
I20260926 13:52:46.466508 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.462361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:42901 (local address 127.10.236.194:33569)
0926 13:52:46.462499 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.462504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.462828 (+ 324us) server_negotiation.cc:407] Connection header received
0926 13:52:46.462949 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.462952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.463009 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.463105 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:46.463651 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.464137 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.465418 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.465872 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.466000 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.466053 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.466253 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.466381 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:52:46.467144 25377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7931987dd14bf08cd960e884320218" candidate_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8582728560249a7a84ceddc1aab2e32" is_pre_election: true
I20260926 13:52:46.467187 25243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7931987dd14bf08cd960e884320218" candidate_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" is_pre_election: true
I20260926 13:52:46.467466 25243 raft_consensus.cc:2454] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50c6ac8632b462b8a04a35471fd0c4d in term 0.
I20260926 13:52:46.467473 25377 raft_consensus.cc:2454] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50c6ac8632b462b8a04a35471fd0c4d in term 0.
I20260926 13:52:46.468003 25042 leader_election.cc:302] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7024a7263cd3483eb765ae7d65cb2ce4, a50c6ac8632b462b8a04a35471fd0c4d; no voters:
I20260926 13:52:46.468248 25448 raft_consensus.cc:2781] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:52:46.468384 25448 raft_consensus.cc:482] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:52:46.468446 25448 raft_consensus.cc:3037] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:46.469595 25448 raft_consensus.cc:504] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.469985 25448 leader_election.cc:288] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 1 election: Requested vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
I20260926 13:52:46.470388 25377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7931987dd14bf08cd960e884320218" candidate_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8582728560249a7a84ceddc1aab2e32"
I20260926 13:52:46.470405 25243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7931987dd14bf08cd960e884320218" candidate_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7024a7263cd3483eb765ae7d65cb2ce4"
I20260926 13:52:46.470496 25377 raft_consensus.cc:3037] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:46.470542 25243 raft_consensus.cc:3037] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:46.471727 25377 raft_consensus.cc:2454] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50c6ac8632b462b8a04a35471fd0c4d in term 1.
I20260926 13:52:46.471843 25243 raft_consensus.cc:2454] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50c6ac8632b462b8a04a35471fd0c4d in term 1.
I20260926 13:52:46.472082 25045 leader_election.cc:302] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50c6ac8632b462b8a04a35471fd0c4d, b8582728560249a7a84ceddc1aab2e32; no voters:
I20260926 13:52:46.472291 25448 raft_consensus.cc:2781] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:46.472487 25448 raft_consensus.cc:686] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 LEADER]: Becoming Leader. State: Replica: a50c6ac8632b462b8a04a35471fd0c4d, State: Running, Role: LEADER
I20260926 13:52:46.472810 25448 consensus_queue.cc:237] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.476269 24964 catalog_manager.cc:5495] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193). New cstate: current_term: 1 leader_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } health_report { overall_health: UNKNOWN } } }
I20260926 13:52:46.517004 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.513340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.10.236.193:38219)
0926 13:52:46.513467 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.513471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.513485 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:52:46.513618 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.513620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.513686 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.513791 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:52:46.514023 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.514530 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.515354 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.515701 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.515831 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.516156 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:52:46.516602 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:52:46.516703 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.516758 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.516810 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:52:46.522858 25243 raft_consensus.cc:1261] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Refusing update from remote peer a50c6ac8632b462b8a04a35471fd0c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:46.522859 25377 raft_consensus.cc:1261] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Refusing update from remote peer a50c6ac8632b462b8a04a35471fd0c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:46.523659 25451 consensus_queue.cc:1035] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:46.523890 25448 consensus_queue.cc:1035] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:46.532097 25456 mvcc.cc:205] Tried to move back new op lower bound from 7333604419670380544 to 7333604419475152896. Current Snapshot: MvccSnapshot[applied={T|T < 7333604419670380544}]
I20260926 13:52:46.532323 25455 mvcc.cc:205] Tried to move back new op lower bound from 7333604419670380544 to 7333604419475152896. Current Snapshot: MvccSnapshot[applied={T|T < 7333604419670380544}]
I20260926 13:52:46.540132 24963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:52:46.540490 24963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:52:46.543006 24963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Sending DeleteTablet for 3 replicas of tablet 8c7931987dd14bf08cd960e884320218
I20260926 13:52:46.543910 25223 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c7931987dd14bf08cd960e884320218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:46450
I20260926 13:52:46.544090 25357 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c7931987dd14bf08cd960e884320218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:34402
I20260926 13:52:46.544263 25089 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c7931987dd14bf08cd960e884320218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:42568
I20260926 13:52:46.544483 25463 tablet_replica.cc:330] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: stopping tablet replica
I20260926 13:52:46.544746 25465 tablet_replica.cc:330] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: stopping tablet replica
I20260926 13:52:46.544822 25463 raft_consensus.cc:2229] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:46.544953 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25026
I20260926 13:52:46.545012 25465 raft_consensus.cc:2229] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:52:46.545811 25463 raft_consensus.cc:2258] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:46.547147 25463 ts_tablet_manager.cc:1905] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:52:46.549558 25463 ts_tablet_manager.cc:1918] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:52:46.549674 25463 log.cc:1196] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/8c7931987dd14bf08cd960e884320218
I20260926 13:52:46.550026 25463 ts_tablet_manager.cc:1939] T 8c7931987dd14bf08cd960e884320218 P 7024a7263cd3483eb765ae7d65cb2ce4: Deleting consensus metadata
I20260926 13:52:46.550055 25464 tablet_replica.cc:330] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: stopping tablet replica
I20260926 13:52:46.550503 25464 raft_consensus.cc:2229] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:46.550890 25464 raft_consensus.cc:2258] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:46.551371 24948 catalog_manager.cc:4841] TS 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): tablet 8c7931987dd14bf08cd960e884320218 (table test-table [id=44596aa849a343f7976695a917d14008]) successfully deleted
I20260926 13:52:46.552306 25464 ts_tablet_manager.cc:1905] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:52:46.554610 25464 ts_tablet_manager.cc:1918] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260926 13:52:46.554651 24949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:38219 (error 108)
I20260926 13:52:46.554711 25464 log.cc:1196] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/8c7931987dd14bf08cd960e884320218
I20260926 13:52:46.555146 25464 ts_tablet_manager.cc:1939] T 8c7931987dd14bf08cd960e884320218 P b8582728560249a7a84ceddc1aab2e32: Deleting consensus metadata
I20260926 13:52:46.555450 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25160
I20260926 13:52:46.557016 24951 catalog_manager.cc:4841] TS b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): tablet 8c7931987dd14bf08cd960e884320218 (table test-table [id=44596aa849a343f7976695a917d14008]) successfully deleted
I20260926 13:52:46.557320 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.557006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:38219 (local address 127.0.0.1:42582)
0926 13:52:46.557147 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.557221 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 13:52:46.557971 24949 catalog_manager.cc:4584] TS a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c7931987dd14bf08cd960e884320218: Network error: Client connection negotiation failed: client connection to 127.10.236.193:38219: connect: Connection refused (error 111)
I20260926 13:52:46.561005 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25294
I20260926 13:52:46.565971 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 24932
I20260926 13:52:46.571803 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43247
--webserver_interface=127.10.236.254
--webserver_port=46051
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:43247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:46.689921 25467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:46.690244 25467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:46.690339 25467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:46.694523 25467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:46.694619 25467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:46.694651 25467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:46.694693 25467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:46.694743 25467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:46.700304 25467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:43247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=46051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:46.701392 25467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:46.702426 25467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:46.716039 25474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.716190 25467 server_base.cc:1034] running on GCE node
W20260926 13:52:46.716059 25473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:46.716039 25476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.716929 25467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:46.718210 25467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:46.719405 25467 hybrid_clock.cc:648] HybridClock initialized: now 1790430766719388 us; error 28 us; skew 500 ppm
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:46.722249 25467 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:46.723218 25467 webserver.cc:460] Webserver started at http://127.10.236.254:46051/ using document root <none> and password file <none>
I20260926 13:52:46.723871 25467 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:46.723980 25467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:46.728022 25467 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 13:52:46.730262 25483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.731683 25467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:46.731813 25467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "6c57251a0e1e45c0a16d51336c79eaad"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "c4fadb97ac4c97fdf64d71fc823258bb"
server_key_iv: "6566f5145c7a9d47173a4a3c427b41a8"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.732434 25467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:46.744091 25467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:46.747257 25467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:46.747454 25467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:46.754768 25467 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:43247
I20260926 13:52:46.755079 25535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:43247 every 8 connection(s)
I20260926 13:52:46.755980 25467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:46.758178 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25467
I20260926 13:52:46.758656 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:38219
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:43247
--webserver_port=34717
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:46.760541 25536 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:52:46.762353 25536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap starting.
I20260926 13:52:46.779711 25536 log.cc:824] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:46.789079 25536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:52:46.789512 25536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap complete.
I20260926 13:52:46.793838 25536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:46.794282 25536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c57251a0e1e45c0a16d51336c79eaad, State: Initialized, Role: FOLLOWER
I20260926 13:52:46.794844 25536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:46.794999 25536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:52:46.795099 25536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:52:46.795226 25536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:52:46.797607 25536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:46.798007 25536 leader_election.cc:302] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c57251a0e1e45c0a16d51336c79eaad; no voters:
I20260926 13:52:46.798391 25536 leader_election.cc:288] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:52:46.798486 25540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:52:46.798712 25540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 LEADER]: Becoming Leader. State: Replica: 6c57251a0e1e45c0a16d51336c79eaad, State: Running, Role: LEADER
I20260926 13:52:46.799021 25540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:46.799738 25536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:46.800064 25542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c57251a0e1e45c0a16d51336c79eaad. Latest consensus state: current_term: 2 leader_uuid: "6c57251a0e1e45c0a16d51336c79eaad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } } }
I20260926 13:52:46.800206 25542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:46.800444 25541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c57251a0e1e45c0a16d51336c79eaad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } } }
I20260926 13:52:46.800544 25541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:46.800593 25544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:46.805682 25544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:46.806552 25544 catalog_manager.cc:1259] Loaded cluster ID: ac4f58942b8743ad9697838ad9210a33
I20260926 13:52:46.806619 25544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:52:46.808115 25544 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 13:52:46.813125 25544 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 13:52:46.818996 25544 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:52:46.819890 25544 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Loaded TSK: 0
I20260926 13:52:46.821113 25544 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:52:46.877882 25538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:46.878196 25538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:46.878274 25538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:46.882519 25538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:46.882642 25538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:46.882745 25538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:52:46.888432 25538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:38219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=34717
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:46.889554 25538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:46.890595 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:46.900589 25564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:46.900601 25563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.902062 25538 server_base.cc:1034] running on GCE node
W20260926 13:52:46.902103 25566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:46.902562 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:46.903429 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:46.904718 25538 hybrid_clock.cc:648] HybridClock initialized: now 1790430766904678 us; error 48 us; skew 500 ppm
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:46.907912 25538 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:52:46.909091 25538 webserver.cc:460] Webserver started at http://127.10.236.193:34717/ using document root <none> and password file <none>
I20260926 13:52:46.909813 25538 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:46.909920 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:46.915233 25538 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:52:46.918143 25573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:46.919809 25538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:46.919951 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "a50c6ac8632b462b8a04a35471fd0c4d"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "d0ebbbedf2feaa3ed99b1522c38f1b97"
server_key_iv: "d672dc2ea463e4cb152bb142d86f3e89"
server_key_version: "encryptionkey@0"
I20260926 13:52:46.920706 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:46.948328 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:46.952494 25538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:46.952715 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:46.953528 25538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:46.954970 25580 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:52:46.959713 25538 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:52:46.959815 25538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260926 13:52:46.959895 25538 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:52:46.962055 25538 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:52:46.962114 25538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 13:52:46.962534 25580 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap starting.
I20260926 13:52:46.973346 25538 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:38219
I20260926 13:52:46.973459 25687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:38219 every 8 connection(s)
I20260926 13:52:46.974414 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:52:46.976074 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25538
I20260926 13:52:46.976502 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:33569
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:43247
--webserver_port=40749
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:46.983971 25580 log.cc:824] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:52:46 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:46.993125 25580 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:52:46.993636 25580 tablet_bootstrap.cc:492] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap complete.
I20260926 13:52:46.995070 25580 ts_tablet_manager.cc:1397] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent bootstrapping tablet: real 0.033s user 0.026s sys 0.003s
I20260926 13:52:46.996204 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.977045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:52953)
0926 13:52:46.978239 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:46.978284 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:46.979127 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:46.980336 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:46.980344 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:46.980869 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:46.981540 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.981562 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.983617 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.983623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.985289 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:46.985305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.986383 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.986390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:46.986628 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.987406 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:46.987433 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:46.989600 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:46.992452 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:46.992458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:46.992472 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:46.992806 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:46.993240 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:46.993243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:46.993245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:46.993405 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:46.994273 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:46.994280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:46.994561 (+ 281us) client_negotiation.cc:769] Sending connection context
0926 13:52:46.994761 (+ 200us) client_negotiation.cc:240] Negotiation successful
0926 13:52:46.995223 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":141,"threads_started":1}
I20260926 13:52:46.996199 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:46.976936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:52953 (local address 127.10.236.254:43247)
0926 13:52:46.977883 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:46.977905 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:46.978419 (+ 514us) server_negotiation.cc:407] Connection header received
0926 13:52:46.979616 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:46.979628 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:46.979943 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:46.980267 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:46.981768 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.983404 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.985441 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:46.986208 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:46.986405 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:46.989846 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:46.989856 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:46.989873 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:46.989913 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:46.992331 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.992941 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.992947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.992953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.993114 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:46.993514 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:46.993521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:46.993527 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:46.994068 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:46.994207 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:46.994905 (+ 698us) server_negotiation.cc:299] Negotiation successful
0926 13:52:46.995232 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":120,"threads_started":1}
I20260926 13:52:46.998592 25580 raft_consensus.cc:348] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:46.999140 25580 raft_consensus.cc:729] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50c6ac8632b462b8a04a35471fd0c4d, State: Initialized, Role: FOLLOWER
I20260926 13:52:46.999696 25580 consensus_queue.cc:260] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } }
I20260926 13:52:47.000197 25688 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43247
I20260926 13:52:47.000430 25688 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:47.000811 25580 ts_tablet_manager.cc:1428] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260926 13:52:47.001066 25688 heartbeater.cc:511] Master 127.10.236.254:43247 requested a full tablet report, sending...
I20260926 13:52:47.003607 25500 ts_manager.cc:194] Registered new tserver with Master: a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219)
I20260926 13:52:47.007668 25500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:52953
I20260926 13:52:47.010262 25688 heartbeater.cc:503] Master 127.10.236.254:43247 was elected leader, sending a full tablet report...
Sep 26 13:52:47 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430766, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:52:47.018822 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.007695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.10.236.193:38219)
0926 13:52:47.008043 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.008054 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.008076 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:52:47.008144 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.008149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.008350 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.008483 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:47.009305 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.010451 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.011833 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.012316 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.012424 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.014854 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:47.014864 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:47.014877 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:47.014903 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:47.016796 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.017347 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.017353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.017355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.017451 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.017756 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.017760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.017762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.018020 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:47.018173 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.018495 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.018657 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":135,"threads_started":1}
I20260926 13:52:47.018832 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.006866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:38219 (local address 127.0.0.1:42592)
0926 13:52:47.007308 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.007357 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.007502 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.008511 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.008518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.009019 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:47.009162 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.009172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.010579 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.010583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.011688 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.011700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.012437 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.012440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.012527 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.013168 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:47.013181 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:47.014708 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:47.016917 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:47.016926 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:47.016948 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:47.017197 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:47.017551 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:47.017555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:47.017557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:47.017659 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:47.018136 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:47.018141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:47.018271 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.018413 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.018629 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":125,"threads_started":1}
I20260926 13:52:47.019922 25622 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c7931987dd14bf08cd960e884320218 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:46 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:42592
I20260926 13:52:47.020910 25697 tablet_replica.cc:330] stopping tablet replica
I20260926 13:52:47.021210 25697 raft_consensus.cc:2229] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:47.021461 25697 raft_consensus.cc:2258] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:47.022426 25697 ts_tablet_manager.cc:1905] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:52:47.026073 25697 ts_tablet_manager.cc:1918] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:52:47.026191 25697 log.cc:1196] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/8c7931987dd14bf08cd960e884320218
I20260926 13:52:47.026454 25697 ts_tablet_manager.cc:1939] T 8c7931987dd14bf08cd960e884320218 P a50c6ac8632b462b8a04a35471fd0c4d: Deleting consensus metadata
I20260926 13:52:47.028400 25485 catalog_manager.cc:4841] TS a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219): tablet 8c7931987dd14bf08cd960e884320218 (table test-table [id=44596aa849a343f7976695a917d14008]) successfully deleted
W20260926 13:52:47.098340 25692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:47.098587 25692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:47.098634 25692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:47.103096 25692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:47.103166 25692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:47.103229 25692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:52:47.108776 25692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:33569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=40749
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:47.109881 25692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:47.110893 25692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:47.120569 25705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:47.120623 25702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:47.120677 25692 server_base.cc:1034] running on GCE node
W20260926 13:52:47.120600 25703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:47.121241 25692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:47.121879 25692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:47.123062 25692 hybrid_clock.cc:648] HybridClock initialized: now 1790430767123048 us; error 27 us; skew 500 ppm
Sep 26 13:52:47 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430767, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:47.125959 25692 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:52:47.126994 25692 webserver.cc:460] Webserver started at http://127.10.236.194:40749/ using document root <none> and password file <none>
I20260926 13:52:47.127681 25692 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:47.127789 25692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:47.131661 25692 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 13:52:47.133770 25712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.135075 25692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:52:47.135223 25692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "7024a7263cd3483eb765ae7d65cb2ce4"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "41572d4ee005d8b2addeade9d0c00e64"
server_key_iv: "2bd7161b0a0c19881ff4fed6e5010ea9"
server_key_version: "encryptionkey@0"
I20260926 13:52:47.135906 25692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:47.154124 25692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:47.157591 25692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:47.157806 25692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:47.158418 25692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:47.159253 25692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:47.159325 25692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.159399 25692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:47.159442 25692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.168505 25692 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33569
I20260926 13:52:47.168542 25825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33569 every 8 connection(s)
I20260926 13:52:47.169507 25692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:52:47.175587 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25692
I20260926 13:52:47.176003 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:37033
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:43247
--webserver_port=33637
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:52:47 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430767, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:47.183543 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.171404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:58305 (local address 127.10.236.254:43247)
0926 13:52:47.171591 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.171594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.172432 (+ 838us) server_negotiation.cc:407] Connection header received
0926 13:52:47.173829 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.173832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.173897 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.173999 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:47.175177 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.175845 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.177080 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.177332 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.177446 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.179946 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:47.179949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:47.179952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:47.179976 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:47.181452 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.181969 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.181974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.181976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.182065 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.182404 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.182409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.182411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.182594 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:47.182702 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.183195 (+ 493us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.183339 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:52:47.184355 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.171848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:58305)
0926 13:52:47.172278 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.172330 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.173608 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.174191 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.174198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.174557 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:47.175030 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.175044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.175965 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.175968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.176957 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.176967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.177485 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.177489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.177665 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.178272 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:47.178287 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:47.179765 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:47.181581 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:47.181586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:47.181601 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:47.181866 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:47.182159 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:47.182163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:47.182165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:47.182310 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:47.182697 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:47.182704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:47.182963 (+ 259us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.183156 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.183485 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":102,"threads_started":1}
I20260926 13:52:47.185613 25826 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43247
I20260926 13:52:47.185869 25826 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:47.186442 25826 heartbeater.cc:511] Master 127.10.236.254:43247 requested a full tablet report, sending...
I20260926 13:52:47.187556 25500 ts_manager.cc:194] Registered new tserver with Master: 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569)
I20260926 13:52:47.188128 25500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:58305
W20260926 13:52:47.295536 25829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:47.295861 25829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:47.295938 25829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:47.300001 25829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:47.300118 25829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:47.300241 25829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:52:47.306043 25829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:37033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=33637
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:47.307168 25829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:47.308385 25829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:47.318466 25835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:47.318544 25837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:47.318449 25834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:47.318638 25829 server_base.cc:1034] running on GCE node
I20260926 13:52:47.319156 25829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:47.319931 25829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:47.321154 25829 hybrid_clock.cc:648] HybridClock initialized: now 1790430767321098 us; error 30 us; skew 500 ppm
Sep 26 13:52:47 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430767, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:47.324251 25829 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:52:47.325205 25829 webserver.cc:460] Webserver started at http://127.10.236.195:33637/ using document root <none> and password file <none>
I20260926 13:52:47.325841 25829 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:47.325965 25829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:47.330111 25829 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:52:47.335328 25844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.337332 25829 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260926 13:52:47.337460 25829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "b8582728560249a7a84ceddc1aab2e32"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "999926a6a00320b53992389cf427398c"
server_key_iv: "90e8642e614b9c636094116ac84e680a"
server_key_version: "encryptionkey@0"
I20260926 13:52:47.338341 25829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:47.357498 25829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:47.361735 25829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:47.361985 25829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:47.362809 25829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:47.363960 25829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:47.364043 25829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.364117 25829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:47.364161 25829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.375617 25829 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:37033
I20260926 13:52:47.375684 25957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:37033 every 8 connection(s)
I20260926 13:52:47.376735 25829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:52:47.382863 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25829
Sep 26 13:52:47 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430767, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:52:47.392390 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.383610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.10.236.254:43247)
0926 13:52:47.384041 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.384045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.384056 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:52:47.384089 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.384092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.384136 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.384226 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:47.384934 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.385624 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.386504 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.386870 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.386985 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.388623 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:47.388628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:47.388631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:47.388676 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:47.390421 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.390932 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.390936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.390938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.391108 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.391436 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.391439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.391440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.391720 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:47.391944 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.392090 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.392197 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":358,"spinlock_wait_cycles":1408,"thread_start_us":83,"threads_started":1}
I20260926 13:52:47.392971 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.379155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:37955 (local address 127.10.236.254:43247)
0926 13:52:47.379325 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.379329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.380302 (+ 973us) server_negotiation.cc:407] Connection header received
0926 13:52:47.381666 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.381669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.381735 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.381817 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:47.383122 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.383784 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.385219 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.385458 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.385556 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.388522 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:47.388526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:47.388528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:47.388555 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:47.390265 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.390932 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.390936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.390938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.391096 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:47.391697 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:47.391701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:47.391703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:47.391893 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:47.391979 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.392666 (+ 687us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.392790 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":4864}
I20260926 13:52:47.393965 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.379634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:37955)
0926 13:52:47.380135 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.380178 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.381432 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.381992 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.381999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.382398 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:47.382879 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.382893 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.383934 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.383938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.385079 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.385090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.385609 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.385613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.385820 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.386487 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:47.386510 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:47.388341 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:47.390387 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:47.390393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:47.390410 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:47.390814 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:47.391224 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:47.391228 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:47.391231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:47.391431 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:47.392003 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:47.392012 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:47.392354 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.392612 (+ 258us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.392991 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260926 13:52:47.395081 25958 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43247
I20260926 13:52:47.395299 25958 heartbeater.cc:464] Registering TS with master...
I20260926 13:52:47.395789 25958 heartbeater.cc:511] Master 127.10.236.254:43247 requested a full tablet report, sending...
I20260926 13:52:47.396775 25500 ts_manager.cc:194] Registered new tserver with Master: b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
I20260926 13:52:47.397360 25500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:37955
I20260926 13:52:47.406087 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:52:47.410660 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.406797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.10.236.254:43247)
0926 13:52:47.406915 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.406919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.407119 (+ 200us) server_negotiation.cc:407] Connection header received
0926 13:52:47.407272 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.407274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.407405 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.407560 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:52:47.407834 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.408413 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.409315 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.409533 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.409614 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.409902 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:52:47.410317 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:52:47.410408 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.410504 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.410549 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:52:47.411324 25500 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:52:47.414748 25500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:52:47.423192 25622 tablet_service.cc:1467] Processing CreateTablet for tablet b0b5f78c8c9e477bbfe910745be03b44 (DEFAULT_TABLE table=test-table [id=bea2d96075404d799cf131644831c091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:47.423981 25622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b5f78c8c9e477bbfe910745be03b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:47.427255 25580 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap starting.
I20260926 13:52:47.427986 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.422047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37033 (local address 127.0.0.1:34410)
0926 13:52:47.422321 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.422337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.422441 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.423238 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.423241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.423258 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:47.423617 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.423621 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.424891 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.424895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.426053 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.426063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.427520 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.427524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.427621 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.427673 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.427735 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.427821 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":83,"thread_start_us":99,"threads_started":1}
I20260926 13:52:47.428329 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.421852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33569 (local address 127.0.0.1:46466)
0926 13:52:47.422117 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.422137 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.422218 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.423347 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.423350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.423371 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:47.423518 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.423528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.425340 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.425344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.426435 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.426443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.427878 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.427881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.427974 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.427992 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.428080 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.428171 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":20}
I20260926 13:52:47.428337 25580 tablet_bootstrap.cc:654] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:47.428635 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.422012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46466 (local address 127.10.236.194:33569)
0926 13:52:47.422744 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.422759 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.422785 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:52:47.422868 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.422874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.423119 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.423555 (+ 436us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:47.423725 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.425183 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.426543 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.427302 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.427472 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.427649 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.428299 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.428494 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":184,"threads_started":1}
I20260926 13:52:47.428804 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.422387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.10.236.195:37033)
0926 13:52:47.422858 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.422867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.422884 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:52:47.422944 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.422948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.423091 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.423261 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:47.423712 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.424712 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.426191 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.427317 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.428235 (+ 918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.428399 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.428454 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.428630 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":155,"threads_started":1}
I20260926 13:52:47.429891 25580 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: No bootstrap required, opened a new log
I20260926 13:52:47.429983 25580 ts_tablet_manager.cc:1397] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:52:47.430428 25580 raft_consensus.cc:348] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.430696 25580 raft_consensus.cc:374] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:47.430742 25580 raft_consensus.cc:729] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50c6ac8632b462b8a04a35471fd0c4d, State: Initialized, Role: FOLLOWER
I20260926 13:52:47.431236 25580 consensus_queue.cc:260] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.431551 25580 ts_tablet_manager.cc:1428] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:52:47.431891 25760 tablet_service.cc:1467] Processing CreateTablet for tablet b0b5f78c8c9e477bbfe910745be03b44 (DEFAULT_TABLE table=test-table [id=bea2d96075404d799cf131644831c091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:47.431896 25892 tablet_service.cc:1467] Processing CreateTablet for tablet b0b5f78c8c9e477bbfe910745be03b44 (DEFAULT_TABLE table=test-table [id=bea2d96075404d799cf131644831c091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:52:47.432890 25892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b5f78c8c9e477bbfe910745be03b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:47.432907 25760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b5f78c8c9e477bbfe910745be03b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:52:47.440341 25971 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Bootstrap starting.
I20260926 13:52:47.440781 25972 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Bootstrap starting.
I20260926 13:52:47.443135 25972 tablet_bootstrap.cc:654] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:47.444156 25972 log.cc:824] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:47.446079 25971 tablet_bootstrap.cc:654] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Neither blocks nor log segments found. Creating new log.
I20260926 13:52:47.446491 25972 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: No bootstrap required, opened a new log
I20260926 13:52:47.446718 25972 ts_tablet_manager.cc:1397] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:52:47.446918 25971 log.cc:824] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:47.448771 25971 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: No bootstrap required, opened a new log
I20260926 13:52:47.448974 25971 ts_tablet_manager.cc:1397] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260926 13:52:47.449792 25972 raft_consensus.cc:348] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.450037 25972 raft_consensus.cc:374] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:47.450098 25972 raft_consensus.cc:729] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7024a7263cd3483eb765ae7d65cb2ce4, State: Initialized, Role: FOLLOWER
I20260926 13:52:47.450492 25972 consensus_queue.cc:260] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.451152 25826 heartbeater.cc:503] Master 127.10.236.254:43247 was elected leader, sending a full tablet report...
I20260926 13:52:47.451444 25972 ts_tablet_manager.cc:1428] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 13:52:47.452793 25971 raft_consensus.cc:348] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.453088 25971 raft_consensus.cc:374] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:52:47.453173 25971 raft_consensus.cc:729] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8582728560249a7a84ceddc1aab2e32, State: Initialized, Role: FOLLOWER
I20260926 13:52:47.453612 25971 consensus_queue.cc:260] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.454442 25958 heartbeater.cc:503] Master 127.10.236.254:43247 was elected leader, sending a full tablet report...
I20260926 13:52:47.454632 25971 ts_tablet_manager.cc:1428] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260926 13:52:47.475668 25689 tablet.cc:2369] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:52:47.524441 25975 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:47.524642 25975 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.525879 25975 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033), a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219)
I20260926 13:52:47.529981 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.526110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37033 (local address 127.10.236.194:35327)
0926 13:52:47.526244 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.526259 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.526352 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.526670 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.526672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.526697 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:47.526833 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.526841 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.527829 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.527832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.528744 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.528752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.529554 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.529557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.529665 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.529691 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.529772 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.529842 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 13:52:47.529978 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.526279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:35327 (local address 127.10.236.195:37033)
0926 13:52:47.526418 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.526423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.526438 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:52:47.526495 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.526498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.526555 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.526641 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:47.527028 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.527727 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.528861 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.529446 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.529536 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.529578 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.529815 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.529882 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:52:47.530491 25977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.526137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:38219 (local address 127.10.236.194:56461)
0926 13:52:47.526528 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.526541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:47.526680 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:47.527034 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:47.527037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:47.527055 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:52:47.527167 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.527174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.528138 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.528140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.529205 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:47.529216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.530087 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.530091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:47.530189 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.530231 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:52:47.530304 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:52:47.530368 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":112,"threads_started":1}
I20260926 13:52:47.530745 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.526342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:56461 (local address 127.10.236.193:38219)
0926 13:52:47.526612 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.526618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.526638 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:52:47.526789 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.526794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.526871 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.527012 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:52:47.527274 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.528033 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.529333 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.529943 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.530346 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.530518 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.530557 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.530620 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
I20260926 13:52:47.530701 25912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b5f78c8c9e477bbfe910745be03b44" candidate_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8582728560249a7a84ceddc1aab2e32" is_pre_election: true
I20260926 13:52:47.531023 25912 raft_consensus.cc:2454] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7024a7263cd3483eb765ae7d65cb2ce4 in term 0.
I20260926 13:52:47.531292 25642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b5f78c8c9e477bbfe910745be03b44" candidate_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" is_pre_election: true
I20260926 13:52:47.531509 25716 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7024a7263cd3483eb765ae7d65cb2ce4, b8582728560249a7a84ceddc1aab2e32; no voters:
I20260926 13:52:47.531577 25642 raft_consensus.cc:2454] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7024a7263cd3483eb765ae7d65cb2ce4 in term 0.
I20260926 13:52:47.531809 25975 raft_consensus.cc:2781] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:52:47.531898 25975 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:52:47.531951 25975 raft_consensus.cc:3037] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:47.532972 25975 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.533314 25975 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [CANDIDATE]: Term 1 election: Requested vote from peers b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033), a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219)
I20260926 13:52:47.533644 25912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b5f78c8c9e477bbfe910745be03b44" candidate_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8582728560249a7a84ceddc1aab2e32"
I20260926 13:52:47.533738 25642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b5f78c8c9e477bbfe910745be03b44" candidate_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50c6ac8632b462b8a04a35471fd0c4d"
I20260926 13:52:47.533785 25912 raft_consensus.cc:3037] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:47.533907 25642 raft_consensus.cc:3037] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:52:47.534863 25912 raft_consensus.cc:2454] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7024a7263cd3483eb765ae7d65cb2ce4 in term 1.
I20260926 13:52:47.534953 25642 raft_consensus.cc:2454] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7024a7263cd3483eb765ae7d65cb2ce4 in term 1.
I20260926 13:52:47.535248 25714 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7024a7263cd3483eb765ae7d65cb2ce4, a50c6ac8632b462b8a04a35471fd0c4d; no voters:
I20260926 13:52:47.535420 25975 raft_consensus.cc:2781] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:52:47.535705 25975 raft_consensus.cc:686] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 LEADER]: Becoming Leader. State: Replica: 7024a7263cd3483eb765ae7d65cb2ce4, State: Running, Role: LEADER
I20260926 13:52:47.536077 25975 consensus_queue.cc:237] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:47.539179 25499 catalog_manager.cc:5495] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } health_report { overall_health: UNKNOWN } } }
I20260926 13:52:47.551457 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.547707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.10.236.194:33569)
0926 13:52:47.548049 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:47.548054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:47.548066 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:52:47.548136 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:47.548138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:47.548188 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:47.548300 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:52:47.548536 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.549094 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.549963 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:47.550224 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:47.550363 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:47.550661 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:52:47.551067 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:52:47.551196 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:47.551258 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 13:52:47.551314 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254}
I20260926 13:52:47.557353 25912 raft_consensus.cc:1261] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Refusing update from remote peer 7024a7263cd3483eb765ae7d65cb2ce4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:47.557350 25642 raft_consensus.cc:1261] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Refusing update from remote peer 7024a7263cd3483eb765ae7d65cb2ce4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:52:47.558007 25975 consensus_queue.cc:1035] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:47.558224 25979 consensus_queue.cc:1035] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:52:47.574225 25500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:52:47.574422 25500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:52:47.576784 25500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Sending DeleteTablet for 3 replicas of tablet b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:47.577492 25760 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b5f78c8c9e477bbfe910745be03b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:47 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:46466
I20260926 13:52:47.577494 25892 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b5f78c8c9e477bbfe910745be03b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:47 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:34410
I20260926 13:52:47.577816 25622 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b5f78c8c9e477bbfe910745be03b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:52:47 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:42592
I20260926 13:52:47.577972 25992 tablet_replica.cc:330] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: stopping tablet replica
I20260926 13:52:47.578176 25993 tablet_replica.cc:330] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: stopping tablet replica
I20260926 13:52:47.578293 25993 raft_consensus.cc:2229] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:47.578328 25992 raft_consensus.cc:2229] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:52:47.578430 25993 raft_consensus.cc:2258] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:47.578467 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25538
I20260926 13:52:47.580358 25991 tablet_replica.cc:330] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: stopping tablet replica
I20260926 13:52:47.580379 25992 raft_consensus.cc:2258] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:47.580667 25991 raft_consensus.cc:2229] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:52:47.580983 25991 raft_consensus.cc:2258] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:52:47.581974 25992 ts_tablet_manager.cc:1905] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:52:47.582278 25991 ts_tablet_manager.cc:1905] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:52:47.584746 25991 ts_tablet_manager.cc:1918] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:52:47.584746 25992 ts_tablet_manager.cc:1918] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:52:47.584848 25991 log.cc:1196] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:47.584848 25992 log.cc:1196] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:47.585202 25991 ts_tablet_manager.cc:1939] T b0b5f78c8c9e477bbfe910745be03b44 P b8582728560249a7a84ceddc1aab2e32: Deleting consensus metadata
I20260926 13:52:47.585202 25992 ts_tablet_manager.cc:1939] T b0b5f78c8c9e477bbfe910745be03b44 P 7024a7263cd3483eb765ae7d65cb2ce4: Deleting consensus metadata
W20260926 13:52:47.586395 25485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:38219 (error 108)
I20260926 13:52:47.586432 25487 catalog_manager.cc:4841] TS b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): tablet b0b5f78c8c9e477bbfe910745be03b44 (table test-table [id=bea2d96075404d799cf131644831c091]) successfully deleted
I20260926 13:52:47.586427 25484 catalog_manager.cc:4841] TS 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): tablet b0b5f78c8c9e477bbfe910745be03b44 (table test-table [id=bea2d96075404d799cf131644831c091]) successfully deleted
I20260926 13:52:47.587586 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:47.587252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:38219 (local address 127.0.0.1:42602)
0926 13:52:47.587386 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:47.587472 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 13:52:47.587618 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25692
W20260926 13:52:47.588434 25485 catalog_manager.cc:4584] TS a50c6ac8632b462b8a04a35471fd0c4d (127.10.236.193:38219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0b5f78c8c9e477bbfe910745be03b44: Network error: Client connection negotiation failed: client connection to 127.10.236.193:38219: connect: Connection refused (error 111)
I20260926 13:52:47.593559 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25829
I20260926 13:52:47.599254 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 25467
I20260926 13:52:47.605742 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43247
--webserver_interface=127.10.236.254
--webserver_port=46051
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:43247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:47.717675 25995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:47.717957 25995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:47.717998 25995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:47.722011 25995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:52:47.722065 25995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:47.722084 25995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:47.722100 25995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:52:47.722164 25995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:52:47.727401 25995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:43247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=46051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:47.728545 25995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:47.729578 25995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:47.738306 26001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:47.738485 25995 server_base.cc:1034] running on GCE node
W20260926 13:52:47.738359 26003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:47.738306 26000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:47.739225 25995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:47.740394 25995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:47.741648 25995 hybrid_clock.cc:648] HybridClock initialized: now 1790430767741631 us; error 28 us; skew 500 ppm
Sep 26 13:52:47 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430767, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:47.744593 25995 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:52:47.745424 25995 webserver.cc:460] Webserver started at http://127.10.236.254:46051/ using document root <none> and password file <none>
I20260926 13:52:47.745945 25995 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:47.746001 25995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:47.749764 25995 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260926 13:52:47.751946 26010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:47.753232 25995 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:52:47.753366 25995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "6c57251a0e1e45c0a16d51336c79eaad"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "c4fadb97ac4c97fdf64d71fc823258bb"
server_key_iv: "6566f5145c7a9d47173a4a3c427b41a8"
server_key_version: "encryptionkey@0"
I20260926 13:52:47.753904 25995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:47.811930 25995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:47.815353 25995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:47.815621 25995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:47.823227 25995 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:43247
I20260926 13:52:47.823632 26062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:43247 every 8 connection(s)
I20260926 13:52:47.824406 25995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:52:47.828954 26063 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:52:47.830655 26063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap starting.
I20260926 13:52:47.833173 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 25995
I20260926 13:52:47.833640 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:38219
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:43247
--webserver_port=34717
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:47.848448 26063 log.cc:824] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:47.861393 26063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:52:47.861852 26063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad: Bootstrap complete.
I20260926 13:52:47.866505 26063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:47.866957 26063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c57251a0e1e45c0a16d51336c79eaad, State: Initialized, Role: FOLLOWER
I20260926 13:52:47.867578 26063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:47.867746 26063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:52:47.867852 26063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:52:47.867991 26063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 2 FOLLOWER]: Advancing to term 3
I20260926 13:52:47.870081 26063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:47.870464 26063 leader_election.cc:302] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c57251a0e1e45c0a16d51336c79eaad; no voters:
I20260926 13:52:47.870862 26063 leader_election.cc:288] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 13:52:47.870966 26067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 3 FOLLOWER]: Leader election won for term 3
I20260926 13:52:47.871186 26067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [term 3 LEADER]: Becoming Leader. State: Replica: 6c57251a0e1e45c0a16d51336c79eaad, State: Running, Role: LEADER
I20260926 13:52:47.871528 26067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } }
I20260926 13:52:47.872264 26063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:52:47.872536 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c57251a0e1e45c0a16d51336c79eaad. Latest consensus state: current_term: 3 leader_uuid: "6c57251a0e1e45c0a16d51336c79eaad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } } }
I20260926 13:52:47.872668 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:47.873021 26068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6c57251a0e1e45c0a16d51336c79eaad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57251a0e1e45c0a16d51336c79eaad" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43247 } } }
I20260926 13:52:47.873189 26068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57251a0e1e45c0a16d51336c79eaad [sys.catalog]: This master's current role is: LEADER
I20260926 13:52:47.874292 26071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:52:47.879977 26071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:52:47.880636 26071 catalog_manager.cc:1259] Loaded cluster ID: ac4f58942b8743ad9697838ad9210a33
I20260926 13:52:47.880697 26071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 13:52:47.881677 26071 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9de18aa9ad google::LogMessage::Fail() at ??:0
@ 0x7f9de18ad88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9de18aa4de google::LogMessage::Flush() at ??:0
@ 0x7f9de18aba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9dec1edfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9dec1eed2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9dec1eb579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9dec22f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9de3afc3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9de3afcc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9de3afe3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9de3aed512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9de51316db start_thread at ??:0
@ 0x7f9de03a861f clone at ??:0
W20260926 13:52:47.976050 26065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:47.976362 26065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:47.976411 26065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:47.980756 26065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:47.980829 26065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:47.980891 26065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:52:47.986481 26065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:38219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=34717
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:47.987732 26065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:47.988824 26065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:47.999083 26107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:47.999109 26105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:47.999083 26104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:47.999431 26065 server_base.cc:1034] running on GCE node
I20260926 13:52:48.000056 26065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:48.001020 26065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:48.002233 26065 hybrid_clock.cc:648] HybridClock initialized: now 1790430768002214 us; error 30 us; skew 500 ppm
Sep 26 13:52:48 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430768, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:48.005754 26065 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:52:48.006721 26065 webserver.cc:460] Webserver started at http://127.10.236.193:34717/ using document root <none> and password file <none>
I20260926 13:52:48.007315 26065 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:48.007411 26065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:48.011618 26065 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:52:48.014040 26114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.015566 26065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:48.015708 26065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "a50c6ac8632b462b8a04a35471fd0c4d"
format_stamp: "Formatted at 2026-09-26 13:52:45 on dist-test-slave-r34j"
server_key: "d0ebbbedf2feaa3ed99b1522c38f1b97"
server_key_iv: "d672dc2ea463e4cb152bb142d86f3e89"
server_key_version: "encryptionkey@0"
I20260926 13:52:48.016278 26065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:48.051640 26065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:48.054876 26065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:48.055092 26065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:48.055800 26065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:48.056959 26121 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:52:48.062963 26065 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:52:48.063069 26065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260926 13:52:48.063165 26065 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:52:48.065080 26065 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:52:48.065155 26065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260926 13:52:48.065450 26121 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap starting.
I20260926 13:52:48.076023 26065 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:38219
I20260926 13:52:48.076321 26228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:38219 every 8 connection(s)
I20260926 13:52:48.077114 26065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:52:48.081667 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26065
I20260926 13:52:48.082077 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:33569
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:43247
--webserver_port=40749
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:52:48.089906 26121 log.cc:824] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:52:48.096963 26121 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:52:48.097359 26121 tablet_bootstrap.cc:492] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Bootstrap complete.
I20260926 13:52:48.098434 26121 ts_tablet_manager.cc:1397] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent bootstrapping tablet: real 0.033s user 0.024s sys 0.004s
I20260926 13:52:48.100961 26121 raft_consensus.cc:348] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:48.101397 26121 raft_consensus.cc:729] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50c6ac8632b462b8a04a35471fd0c4d, State: Initialized, Role: FOLLOWER
I20260926 13:52:48.101893 26121 consensus_queue.cc:260] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:48.102851 26121 ts_tablet_manager.cc:1428] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
W20260926 13:52:48.125270 26231 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:52:48.079076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:58565)
0926 13:52:48.079602 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.079655 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:48.080481 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:48.124502 (+ 44021us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":152,"threads_started":1}
W20260926 13:52:48.125916 26229 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:43247 (0 consecutive failures): Network error: Failed to ping master at 127.10.236.254:43247: Client connection negotiation failed: client connection to 127.10.236.254:43247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:52:48.126725 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.126448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:54229)
0926 13:52:48.126574 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.126649 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 13:52:48.127458 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.127217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:41989)
0926 13:52:48.127327 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.127373 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:52:48.127723 26229 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:52:48.209395 26232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:48.209650 26232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:48.209694 26232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:48.214222 26232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:48.214293 26232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:48.214406 26232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:52:48.219859 26232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:33569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=40749
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:48.220980 26232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:48.222075 26232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:48.231845 26239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:48.231850 26240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:48.231869 26242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:48.232263 26232 server_base.cc:1034] running on GCE node
I20260926 13:52:48.232657 26232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:48.233296 26232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:48.234489 26232 hybrid_clock.cc:648] HybridClock initialized: now 1790430768234460 us; error 49 us; skew 500 ppm
Sep 26 13:52:48 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430768, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:48.237349 26232 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:52:48.238169 26232 webserver.cc:460] Webserver started at http://127.10.236.194:40749/ using document root <none> and password file <none>
I20260926 13:52:48.238684 26232 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:48.238741 26232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:48.242676 26232 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:52:48.244809 26249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.246061 26232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:52:48.246181 26232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "7024a7263cd3483eb765ae7d65cb2ce4"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "41572d4ee005d8b2addeade9d0c00e64"
server_key_iv: "2bd7161b0a0c19881ff4fed6e5010ea9"
server_key_version: "encryptionkey@0"
I20260926 13:52:48.246719 26232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:48.267576 26232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:48.270654 26232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:48.270891 26232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:48.271582 26232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:48.272429 26232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:48.272476 26232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.272543 26232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:48.272585 26232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.282107 26232 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:33569
I20260926 13:52:48.282128 26362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:33569 every 8 connection(s)
I20260926 13:52:48.283149 26232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:52:48.287040 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.285427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:53417)
0926 13:52:48.285905 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.286358 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
W20260926 13:52:48.287541 26363 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:43247 (0 consecutive failures): Network error: Failed to ping master at 127.10.236.254:43247: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
I20260926 13:52:48.288344 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.288048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:35615)
0926 13:52:48.288243 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.288289 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 13:52:48.289003 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.288789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:33357)
0926 13:52:48.288905 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.288958 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 13:52:48.289189 26363 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:52:48.292874 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26232
I20260926 13:52:48.293318 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:37033
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:43247
--webserver_port=33637
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:52:48.408377 26366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:52:48.408666 26366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:52:48.408753 26366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:52:48.412977 26366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:52:48.413053 26366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:52:48.413115 26366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:52:48.418643 26366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:37033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=33637
--tserver_master_addrs=127.10.236.254:43247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:52:48.419950 26366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:52:48.421030 26366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:52:48.432426 26374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:48.432605 26372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:52:48.432662 26371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:52:48.432891 26366 server_base.cc:1034] running on GCE node
I20260926 13:52:48.433388 26366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:52:48.434132 26366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:52:48.435356 26366 hybrid_clock.cc:648] HybridClock initialized: now 1790430768435335 us; error 34 us; skew 500 ppm
Sep 26 13:52:48 dist-test-slave-r34j krb5kdc[24919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430768, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:52:48.438508 26366 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:52:48.439636 26366 webserver.cc:460] Webserver started at http://127.10.236.195:33637/ using document root <none> and password file <none>
I20260926 13:52:48.440382 26366 fs_manager.cc:359] Metadata directory not provided
I20260926 13:52:48.440508 26366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:52:48.444844 26366 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:52:48.447356 26381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.448843 26366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:52:48.449014 26366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "b8582728560249a7a84ceddc1aab2e32"
format_stamp: "Formatted at 2026-09-26 13:52:46 on dist-test-slave-r34j"
server_key: "999926a6a00320b53992389cf427398c"
server_key_iv: "90e8642e614b9c636094116ac84e680a"
server_key_version: "encryptionkey@0"
I20260926 13:52:48.449753 26366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:52:48.460415 26366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:52:48.464088 26366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:52:48.464255 26366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:52:48.464804 26366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:52:48.465860 26366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:52:48.465934 26366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.465981 26366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:52:48.465999 26366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:52:48.476759 26366 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:37033
I20260926 13:52:48.476779 26494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:37033 every 8 connection(s)
I20260926 13:52:48.477955 26366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:52:48.480664 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26366
I20260926 13:52:48.482544 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.480342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:50379)
0926 13:52:48.480924 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.481464 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":120,"threads_started":1}
W20260926 13:52:48.483286 26495 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:43247 (0 consecutive failures): Network error: Failed to ping master at 127.10.236.254:43247: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
I20260926 13:52:48.484112 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.483844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:56341)
0926 13:52:48.483999 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.484052 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 13:52:48.485996 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:48.484838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:56405)
0926 13:52:48.485897 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:48.485942 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":960}
W20260926 13:52:48.486282 26495 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:52:49.128934 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.128444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:39197)
0926 13:52:49.128747 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:49.128815 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
I20260926 13:52:49.290189 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.289769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:43437)
0926 13:52:49.290022 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:49.290079 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260926 13:52:49.292518 26501 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:49.292729 26501 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:49.293879 26501 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
Sep 26 13:52:49 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430768, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.194@KRBTEST.COM
Sep 26 13:52:49 dist-test-slave-r34j krb5kdc[24919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430768, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.195@KRBTEST.COM
I20260926 13:52:49.309252 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.294072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:37033 (local address 127.10.236.193:45479)
0926 13:52:49.294709 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:49.294725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:49.294851 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:49.296205 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:49.296208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:49.296732 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:49.297172 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:49.297188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.298435 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.298440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:49.299316 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:49.299325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.300085 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.300088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:49.300314 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:49.300948 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:49.300963 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:49.303013 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:49.305656 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:49.305664 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:49.305681 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:49.306310 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:49.306804 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:49.306808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:49.306810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:49.307044 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:49.307929 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:49.307937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:49.308316 (+ 379us) client_negotiation.cc:769] Sending connection context
0926 13:52:49.308588 (+ 272us) client_negotiation.cc:240] Negotiation successful
0926 13:52:49.309036 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":179,"spinlock_wait_cycles":58368,"thread_start_us":57,"threads_started":1}
I20260926 13:52:49.309275 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.294289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57373 (local address 127.10.236.194:33569)
0926 13:52:49.294668 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:49.294692 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:49.294735 (+ 43us) server_negotiation.cc:407] Connection header received
0926 13:52:49.295417 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:49.295423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:49.295852 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:49.296152 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:49.297359 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.298288 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.299437 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.299957 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.300137 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:49.302892 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:49.302897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:49.302909 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:49.302936 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:49.305475 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:49.306476 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:49.306482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:49.306488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:49.306657 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:49.307167 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:49.307170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:49.307172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:49.307742 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:49.307877 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:49.308800 (+ 923us) server_negotiation.cc:299] Negotiation successful
0926 13:52:49.309082 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":76,"threads_started":1}
I20260926 13:52:49.309252 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.294054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:33569 (local address 127.10.236.193:57373)
0926 13:52:49.294228 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:49.294244 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:52:49.294355 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:52:49.296202 (+ 1847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:52:49.296208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:52:49.296723 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:52:49.297172 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:49.297186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.298444 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.298447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:49.299313 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:52:49.299322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.300084 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.300088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:52:49.300306 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:49.301014 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:52:49.301025 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:52:49.302623 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:52:49.305656 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:49.305664 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:49.305681 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:49.306359 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:49.306804 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:52:49.306808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:52:49.306810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:52:49.307044 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:52:49.307929 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:52:49.307937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:52:49.308316 (+ 379us) client_negotiation.cc:769] Sending connection context
0926 13:52:49.308599 (+ 283us) client_negotiation.cc:240] Negotiation successful
0926 13:52:49.309090 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":330,"spinlock_wait_cycles":29824}
I20260926 13:52:49.309275 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.294486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:45479 (local address 127.10.236.195:37033)
0926 13:52:49.294820 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0926 13:52:49.294833 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:52:49.294877 (+ 44us) server_negotiation.cc:407] Connection header received
0926 13:52:49.295650 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:52:49.295657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:52:49.295971 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:52:49.296258 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:52:49.297346 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.298304 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.299444 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:52:49.299957 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:52:49.300147 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:52:49.303222 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:52:49.303230 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:52:49.303246 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:52:49.303286 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:52:49.305487 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:49.306476 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:49.306482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:49.306488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:49.306656 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:52:49.307167 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:52:49.307170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:52:49.307172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:52:49.307742 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:52:49.307877 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 13:52:49.308769 (+ 892us) server_negotiation.cc:299] Negotiation successful
0926 13:52:49.309082 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":114,"threads_started":1}
W20260926 13:52:49.312108 26115 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
W20260926 13:52:49.312510 26118 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:49.312583 26118 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a50c6ac8632b462b8a04a35471fd0c4d; no voters: 7024a7263cd3483eb765ae7d65cb2ce4, b8582728560249a7a84ceddc1aab2e32
I20260926 13:52:49.312812 26501 raft_consensus.cc:2726] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:52:49.487341 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:49.486897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:60521)
0926 13:52:49.487150 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:49.487211 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
I20260926 13:52:50.130383 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:50.129878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:51467)
0926 13:52:50.130208 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:50.130279 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":84,"threads_started":1}
I20260926 13:52:50.291429 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:50.290964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:55587)
0926 13:52:50.291239 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:50.291338 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260926 13:52:50.488559 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:50.488097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:40747)
0926 13:52:50.488385 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:50.488441 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":72,"threads_started":1}
I20260926 13:52:51.131825 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:51.131267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:46751)
0926 13:52:51.131650 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:51.131722 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":77,"threads_started":1}
I20260926 13:52:51.178692 26510 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:51.178848 26510 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:51.179379 26510 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
W20260926 13:52:51.180327 26118 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
W20260926 13:52:51.180516 26115 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:51.180589 26115 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a50c6ac8632b462b8a04a35471fd0c4d; no voters: 7024a7263cd3483eb765ae7d65cb2ce4, b8582728560249a7a84ceddc1aab2e32
I20260926 13:52:51.180768 26510 raft_consensus.cc:2726] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:52:51.292747 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:51.292304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:46233)
0926 13:52:51.292574 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:51.292627 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260926 13:52:51.489867 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:51.489422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:41513)
0926 13:52:51.489704 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:51.489771 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
I20260926 13:52:52.133118 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:52.132685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:39345)
0926 13:52:52.132967 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:52.133022 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
I20260926 13:52:52.294036 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:52.293579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:44943)
0926 13:52:52.293878 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:52.293939 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":103,"threads_started":1}
I20260926 13:52:52.491204 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:52.490725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:34095)
0926 13:52:52.491025 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:52.491090 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":76,"threads_started":1}
I20260926 13:52:53.134673 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:53.134144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:58363)
0926 13:52:53.134519 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:53.134578 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":105,"threads_started":1}
I20260926 13:52:53.295254 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:53.294836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:38321)
0926 13:52:53.295092 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:53.295148 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
I20260926 13:52:53.492605 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:53.492042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:40423)
0926 13:52:53.492431 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:53.492504 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":132,"threads_started":1}
I20260926 13:52:54.069962 26519 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:54.070128 26519 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:54.070717 26519 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
W20260926 13:52:54.071590 26118 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
W20260926 13:52:54.071779 26115 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:54.071870 26115 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a50c6ac8632b462b8a04a35471fd0c4d; no voters: 7024a7263cd3483eb765ae7d65cb2ce4, b8582728560249a7a84ceddc1aab2e32
I20260926 13:52:54.072047 26519 raft_consensus.cc:2726] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:52:54.136089 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:54.135638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:59407)
0926 13:52:54.135935 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:54.135993 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260926 13:52:54.296685 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:54.296157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:40483)
0926 13:52:54.296478 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:54.296540 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":78,"threads_started":1}
I20260926 13:52:54.493897 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:54.493461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:59245)
0926 13:52:54.493735 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:54.493795 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260926 13:52:55.137480 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:55.137020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:47663)
0926 13:52:55.137312 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:55.137376 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260926 13:52:55.298067 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:55.297571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:36397)
0926 13:52:55.297907 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:55.297970 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":82,"threads_started":1}
I20260926 13:52:55.495293 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:55.494806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:57359)
0926 13:52:55.495138 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:55.495198 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":77,"threads_started":1}
I20260926 13:52:56.095961 26526 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:56.096097 26526 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:56.096575 26526 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
W20260926 13:52:56.097429 26115 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
W20260926 13:52:56.097520 26118 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:56.097563 26115 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a50c6ac8632b462b8a04a35471fd0c4d; no voters: 7024a7263cd3483eb765ae7d65cb2ce4, b8582728560249a7a84ceddc1aab2e32
I20260926 13:52:56.097704 26526 raft_consensus.cc:2726] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:52:56.138827 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:56.138347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:37123)
0926 13:52:56.138644 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:56.138704 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":97,"threads_started":1}
I20260926 13:52:56.299350 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:56.298865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:60319)
0926 13:52:56.299160 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:56.299234 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260926 13:52:56.496697 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:56.496191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:58761)
0926 13:52:56.496527 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:56.496588 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":121,"threads_started":1}
I20260926 13:52:57.140386 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:57.139879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:49323)
0926 13:52:57.140187 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:57.140260 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260926 13:52:57.300661 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:57.300226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:47851)
0926 13:52:57.300505 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:57.300566 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260926 13:52:57.498028 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:57.497596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:39705)
0926 13:52:57.497858 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:57.497924 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260926 13:52:58.141983 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:58.141490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:60759)
0926 13:52:58.141814 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:58.141887 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260926 13:52:58.302130 26534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:58.301659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:36001)
0926 13:52:58.301966 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:58.302031 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
I20260926 13:52:58.499320 26535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:58.498906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:49295)
0926 13:52:58.499172 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:58.499225 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
I20260926 13:52:58.883236 26536 raft_consensus.cc:482] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:52:58.883421 26536 raft_consensus.cc:504] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7024a7263cd3483eb765ae7d65cb2ce4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 33569 } } peers { permanent_uuid: "b8582728560249a7a84ceddc1aab2e32" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 37033 } } peers { permanent_uuid: "a50c6ac8632b462b8a04a35471fd0c4d" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 38219 } }
I20260926 13:52:58.883981 26536 leader_election.cc:288] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569), b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033)
W20260926 13:52:58.884848 26115 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7024a7263cd3483eb765ae7d65cb2ce4 (127.10.236.194:33569): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
W20260926 13:52:58.884984 26118 leader_election.cc:341] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8582728560249a7a84ceddc1aab2e32 (127.10.236.195:37033): Not found: Tablet not found: b0b5f78c8c9e477bbfe910745be03b44
I20260926 13:52:58.885138 26118 leader_election.cc:302] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a50c6ac8632b462b8a04a35471fd0c4d; no voters: 7024a7263cd3483eb765ae7d65cb2ce4, b8582728560249a7a84ceddc1aab2e32
I20260926 13:52:58.885308 26536 raft_consensus.cc:2726] T b0b5f78c8c9e477bbfe910745be03b44 P a50c6ac8632b462b8a04a35471fd0c4d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:52:59.143296 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:59.142872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:33421)
0926 13:52:59.143136 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:59.143194 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260926 13:52:59.303599 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:59.303067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:54343)
0926 13:52:59.303330 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:59.303392 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260926 13:52:59.500823 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:52:59.500207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:49201)
0926 13:52:59.500616 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 13:52:59.500699 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":147,"threads_started":1}
I20260926 13:53:00.144809 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:00.144292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:42385)
0926 13:53:00.144620 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:00.144702 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":89,"threads_started":1}
I20260926 13:53:00.304910 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:00.304459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:34739)
0926 13:53:00.304737 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:00.304803 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260926 13:53:00.502031 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:00.501621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:58875)
0926 13:53:00.501879 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:00.501931 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":95,"threads_started":1}
I20260926 13:53:01.146174 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:01.145721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:48207)
0926 13:53:01.146018 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:01.146077 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260926 13:53:01.306334 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:01.305798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:50039)
0926 13:53:01.306121 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:01.306190 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":136,"threads_started":1}
I20260926 13:53:01.503573 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:01.502919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:40785)
0926 13:53:01.503275 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:01.503419 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":100,"threads_started":1}
I20260926 13:53:02.147599 26546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:02.147031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:52021)
0926 13:53:02.147317 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:02.147423 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":77,"threads_started":1}
I20260926 13:53:02.307807 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:02.307295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:37505)
0926 13:53:02.307610 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:02.307697 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":137,"threads_started":1}
I20260926 13:53:02.504892 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:02.504418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.195:40007)
0926 13:53:02.504711 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:02.504772 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260926 13:53:03.149255 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:03.148769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.193:40833)
0926 13:53:03.149091 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:03.149159 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":76,"threads_started":1}
I20260926 13:53:03.309322 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:03.308771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43247 (local address 127.10.236.194:34735)
0926 13:53:03.309103 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:03.309178 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
I20260926 13:53:03.488816 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26065
I20260926 13:53:03.495817 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26232
I20260926 13:53:03.503695 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26366
2026-09-26T13:53:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19317 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:53:03 dist-test-slave-r34j krb5kdc[26553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:53:03 dist-test-slave-r34j krb5kdc[26553](info): set up 2 sockets
Sep 26 13:53:03 dist-test-slave-r34j krb5kdc[26553](info): commencing operation
krb5kdc: starting...
W20260926 13:53:04.567794 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:53:04 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:53:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:53:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:53:04.703109 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44051
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:04.815791 26566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:04.816126 26566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:04.816202 26566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:04.820328 26566 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 13:53:04.820413 26566 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 13:53:04.820441 26566 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 13:53:04.820472 26566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:53:04.820518 26566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:04.820564 26566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:04.820617 26566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:53:04.820658 26566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:53:04.826104 26566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:04.827262 26566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:04.828357 26566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:04.837347 26572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:04.837347 26574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:04.837342 26571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:04.837404 26566 server_base.cc:1034] running on GCE node
I20260926 13:53:04.838106 26566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:04.839237 26566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:04.840416 26566 hybrid_clock.cc:648] HybridClock initialized: now 1790430784840386 us; error 51 us; skew 500 ppm
Sep 26 13:53:04 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430784, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:04.843149 26566 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:53:04.844122 26566 webserver.cc:460] Webserver started at http://127.10.236.254:42955/ using document root <none> and password file <none>
I20260926 13:53:04.844689 26566 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:04.844775 26566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:04.845005 26566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:53:04.847050 26566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "a3dbd26d7356439da0bfcc43e68709e5"
format_stamp: "Formatted at 2026-09-26 13:53:04 on dist-test-slave-r34j"
server_key: "4ddbc759f7266dbc384bd8f84804dc7f"
server_key_iv: "2aac666281c1bf9f46f139462acecde6"
server_key_version: "encryptionkey@0"
I20260926 13:53:04.847693 26566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "a3dbd26d7356439da0bfcc43e68709e5"
format_stamp: "Formatted at 2026-09-26 13:53:04 on dist-test-slave-r34j"
server_key: "4ddbc759f7266dbc384bd8f84804dc7f"
server_key_iv: "2aac666281c1bf9f46f139462acecde6"
server_key_version: "encryptionkey@0"
I20260926 13:53:04.851649 26566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:53:04.854400 26581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:04.855929 26566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:53:04.856074 26566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "a3dbd26d7356439da0bfcc43e68709e5"
format_stamp: "Formatted at 2026-09-26 13:53:04 on dist-test-slave-r34j"
server_key: "4ddbc759f7266dbc384bd8f84804dc7f"
server_key_iv: "2aac666281c1bf9f46f139462acecde6"
server_key_version: "encryptionkey@0"
I20260926 13:53:04.856197 26566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:04.871677 26566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:04.875113 26566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:04.875326 26566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:04.882706 26566 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44051
I20260926 13:53:04.883020 26633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44051 every 8 connection(s)
I20260926 13:53:04.883764 26566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:53:04.886647 26634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:04.889925 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26566
I20260926 13:53:04.890090 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:53:04.890370 11187 external_mini_cluster.cc:1428] Setting key 67f1ed73dd0c47961261f2d2622ef655
I20260926 13:53:04.892810 26634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap starting.
I20260926 13:53:04.895601 26634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:04.896469 26634 log.cc:824] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:04.898582 26634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: No bootstrap required, opened a new log
Sep 26 13:53:04 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:53:04.902691 26634 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:04.903010 26634 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:04.903059 26634 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3dbd26d7356439da0bfcc43e68709e5, State: Initialized, Role: FOLLOWER
I20260926 13:53:04.903589 26634 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:04.903755 26634 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:53:04.903841 26634 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:53:04.903975 26634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:04.905145 26634 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:04.905543 26634 leader_election.cc:302] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3dbd26d7356439da0bfcc43e68709e5; no voters:
I20260926 13:53:04.905893 26634 leader_election.cc:288] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:53:04.906062 26639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:53:04.906342 26639 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 LEADER]: Becoming Leader. State: Replica: a3dbd26d7356439da0bfcc43e68709e5, State: Running, Role: LEADER
I20260926 13:53:04.906787 26639 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:04.907056 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:04.891869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.10.236.254:44051)
0926 13:53:04.892314 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:04.892332 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:04.892375 (+ 43us) server_negotiation.cc:407] Connection header received
0926 13:53:04.893221 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:04.893243 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:04.893668 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:04.894039 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:04.894730 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:04.896215 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:04.897337 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:04.897992 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:04.898149 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:04.901004 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:04.901010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:04.901028 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:04.901063 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:04.903627 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:04.904183 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:04.904192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:04.904201 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:04.904342 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:04.904652 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:04.904655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:04.904657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:04.905204 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:04.905406 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:04.905697 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 13:53:04.906019 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":112,"threads_started":1}
I20260926 13:53:04.907330 26634 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:53:04.908558 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3dbd26d7356439da0bfcc43e68709e5. Latest consensus state: current_term: 1 leader_uuid: "a3dbd26d7356439da0bfcc43e68709e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } } }
I20260926 13:53:04.908593 26640 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3dbd26d7356439da0bfcc43e68709e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } } }
I20260926 13:53:04.908703 26640 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:53:04.908703 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: This master's current role is: LEADER
W20260926 13:53:04.912468 26654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:53:04.912561 26654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:53:04.912668 26648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:53:04.913682 26648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:53:04.919776 26648 catalog_manager.cc:1347] Generated new cluster ID: 41b1b388b154484fbccf886c866228de
I20260926 13:53:04.919862 26648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:53:04.930150 26648 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:53:04.931131 26648 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:53:04.937637 26648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Generated new TSK 0
I20260926 13:53:04.938704 26648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:53:04.976152 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44051
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:05.104714 26660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:05.105003 26660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:05.105049 26660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:05.109484 26660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:05.109573 26660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:05.109684 26660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:53:05.115044 26660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:05.116171 26660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:05.117190 26660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:05.127840 26668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:05.127844 26666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:05.127899 26660 server_base.cc:1034] running on GCE node
W20260926 13:53:05.127923 26665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:05.128639 26660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:05.129299 26660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:05.130558 26660 hybrid_clock.cc:648] HybridClock initialized: now 1790430785130530 us; error 43 us; skew 500 ppm
Sep 26 13:53:05 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430785, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:05.133546 26660 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:53:05.134471 26660 webserver.cc:460] Webserver started at http://127.10.236.193:39555/ using document root <none> and password file <none>
I20260926 13:53:05.135011 26660 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:05.135068 26660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:05.135233 26660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:53:05.137305 26660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "2773e3cc6ad143bd8fae5ac104378c8b"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "01a9895ba57a65e9a8e57b42aa91cfeb"
server_key_iv: "75121fa2c3d5831de6ac26c44ea804f7"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.137835 26660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "2773e3cc6ad143bd8fae5ac104378c8b"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "01a9895ba57a65e9a8e57b42aa91cfeb"
server_key_iv: "75121fa2c3d5831de6ac26c44ea804f7"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.141806 26660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:53:05.144492 26675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.145972 26660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:53:05.146085 26660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2773e3cc6ad143bd8fae5ac104378c8b"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "01a9895ba57a65e9a8e57b42aa91cfeb"
server_key_iv: "75121fa2c3d5831de6ac26c44ea804f7"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.146179 26660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:05.168373 26660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:05.171605 26660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:05.171762 26660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:05.172335 26660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:05.173256 26660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:53:05.173305 26660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.173379 26660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:05.173412 26660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.182582 26660 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:35817
I20260926 13:53:05.182607 26788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:35817 every 8 connection(s)
I20260926 13:53:05.183511 26660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:53:05.183892 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26660
I20260926 13:53:05.184020 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:53:05.184301 11187 external_mini_cluster.cc:1428] Setting key 2b83a3718f504fc382cf516880bbe5c1
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Sep 26 13:53:05 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430785, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:53:05.198324 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.185590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57707 (local address 127.10.236.254:44051)
0926 13:53:05.185727 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.185733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.186816 (+ 1083us) server_negotiation.cc:407] Connection header received
0926 13:53:05.188252 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.188256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.188311 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.188392 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:05.189579 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.190122 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.191264 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.191570 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.191682 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.194383 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:05.194386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:05.194388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:05.194412 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:05.195912 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.196614 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.196621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.196626 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.196720 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.197097 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.197103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.197108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.197382 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:05.197495 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.197996 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.198142 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:53:05.199156 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.186140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:57707)
0926 13:53:05.186665 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:05.186709 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:05.188048 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:05.188574 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:05.188581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:05.188955 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:05.189431 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.189445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.190256 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.190264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.191135 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.191144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.191728 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.191732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.191903 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.192540 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:05.192557 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:05.194207 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:05.196047 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:05.196052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:05.196065 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:05.196484 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:05.196862 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:05.196864 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:05.196865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:05.196991 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:05.197501 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:05.197505 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:05.197735 (+ 230us) client_negotiation.cc:769] Sending connection context
0926 13:53:05.197953 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 13:53:05.198280 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":144,"threads_started":1}
I20260926 13:53:05.200585 26789 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44051
I20260926 13:53:05.200852 26789 heartbeater.cc:464] Registering TS with master...
I20260926 13:53:05.201350 26789 heartbeater.cc:511] Master 127.10.236.254:44051 requested a full tablet report, sending...
I20260926 13:53:05.203125 26598 ts_manager.cc:194] Registered new tserver with Master: 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817)
I20260926 13:53:05.204838 26598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:57707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:53:05.215669 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44051
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:05.325199 26794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:05.325477 26794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:05.325522 26794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:05.329614 26794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:05.329669 26794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:05.329756 26794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:53:05.335089 26794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:05.336174 26794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:05.337170 26794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:53:05.347127 26794 server_base.cc:1034] running on GCE node
W20260926 13:53:05.347198 26802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:05.347178 26800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:05.347151 26799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:05.347797 26794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:05.348472 26794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:05.349692 26794 hybrid_clock.cc:648] HybridClock initialized: now 1790430785349647 us; error 57 us; skew 500 ppm
Sep 26 13:53:05 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430785, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:05.352799 26794 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:53:05.353718 26794 webserver.cc:460] Webserver started at http://127.10.236.194:42661/ using document root <none> and password file <none>
I20260926 13:53:05.354290 26794 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:05.354394 26794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:05.354643 26794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:53:05.356737 26794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "359713cadb03402a9e96cd6f9416e2ee"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "8d00e1ca0d1f2e8ccff4ae3e97deda5f"
server_key_iv: "615100bdf380ab5969a430b2ee8c9f2a"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.357300 26794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "359713cadb03402a9e96cd6f9416e2ee"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "8d00e1ca0d1f2e8ccff4ae3e97deda5f"
server_key_iv: "615100bdf380ab5969a430b2ee8c9f2a"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.361255 26794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 13:53:05.363854 26809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.365108 26794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:53:05.365293 26794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "359713cadb03402a9e96cd6f9416e2ee"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "8d00e1ca0d1f2e8ccff4ae3e97deda5f"
server_key_iv: "615100bdf380ab5969a430b2ee8c9f2a"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.365438 26794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:05.390372 26794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:05.393571 26794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:05.393790 26794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:05.394387 26794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:05.395229 26794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:53:05.395299 26794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.395368 26794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:05.395419 26794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.404826 26794 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:44475
I20260926 13:53:05.404844 26922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:44475 every 8 connection(s)
I20260926 13:53:05.405745 26794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:53:05.412514 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26794
I20260926 13:53:05.412654 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:53:05.412949 11187 external_mini_cluster.cc:1428] Setting key a72acbe0273504a6e5de8414bdf4f075
Sep 26 13:53:05 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430785, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:53:05.420946 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.407633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:46231 (local address 127.10.236.254:44051)
0926 13:53:05.407819 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.407823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.408845 (+ 1022us) server_negotiation.cc:407] Connection header received
0926 13:53:05.410368 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.410371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.410427 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.410514 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:05.411691 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.412247 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.413758 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.414067 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.414183 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.417211 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:05.417215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:05.417217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:05.417240 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:05.418701 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.419231 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.419235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.419237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.419318 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.419742 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.419745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.419747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.419947 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:05.420056 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.420580 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.420722 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":83}
I20260926 13:53:05.421778 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.408151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:46231)
0926 13:53:05.408705 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:05.408747 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:05.410146 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:05.410665 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:05.410672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:05.411011 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:05.411527 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.411549 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.412375 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.412379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.413623 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.413632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.414219 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.414223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.414435 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.415109 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:05.415134 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:05.417036 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:05.418838 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:05.418844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:05.418859 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:05.419121 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:05.419413 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:05.419417 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:05.419419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:05.419640 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:05.420054 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:05.420061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:05.420331 (+ 270us) client_negotiation.cc:769] Sending connection context
0926 13:53:05.420524 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 13:53:05.420866 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:53:05.423036 26923 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44051
I20260926 13:53:05.423273 26923 heartbeater.cc:464] Registering TS with master...
I20260926 13:53:05.423916 26923 heartbeater.cc:511] Master 127.10.236.254:44051 requested a full tablet report, sending...
I20260926 13:53:05.425034 26598 ts_manager.cc:194] Registered new tserver with Master: 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:53:05.425899 26598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:46231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:53:05.442663 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44051
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:05.557828 26928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:05.558135 26928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:05.558205 26928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:05.562458 26928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:05.562546 26928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:05.562630 26928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:53:05.567905 26928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:05.569016 26928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:05.570122 26928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:53:05.579869 26928 server_base.cc:1034] running on GCE node
W20260926 13:53:05.579840 26934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:05.579833 26933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:05.580054 26936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:05.580647 26928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:05.581244 26928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:05.582568 26928 hybrid_clock.cc:648] HybridClock initialized: now 1790430785582544 us; error 35 us; skew 500 ppm
Sep 26 13:53:05 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430785, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:05.585323 26928 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:53:05.586196 26928 webserver.cc:460] Webserver started at http://127.10.236.195:42827/ using document root <none> and password file <none>
I20260926 13:53:05.586714 26928 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:05.586769 26928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:05.586931 26928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:53:05.588946 26928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "3782b56c1afa48da8461fd773ff1b746"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "7c79b7e85c1afb7068618e7f5765fae1"
server_key_iv: "fbb69a86152f18f50609eafbf0e31fc6"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.589453 26928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "3782b56c1afa48da8461fd773ff1b746"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "7c79b7e85c1afb7068618e7f5765fae1"
server_key_iv: "fbb69a86152f18f50609eafbf0e31fc6"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.593168 26928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 13:53:05.595690 26943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.597038 26928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:53:05.597175 26928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "3782b56c1afa48da8461fd773ff1b746"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "7c79b7e85c1afb7068618e7f5765fae1"
server_key_iv: "fbb69a86152f18f50609eafbf0e31fc6"
server_key_version: "encryptionkey@0"
I20260926 13:53:05.597307 26928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:05.634927 26928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:05.638132 26928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:05.638341 26928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:05.638942 26928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:05.639914 26928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:53:05.639990 26928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.640065 26928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:05.640110 26928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:05.649121 26928 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:34963
I20260926 13:53:05.649150 27056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:34963 every 8 connection(s)
I20260926 13:53:05.650170 26928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:53:05.659933 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 26928
I20260926 13:53:05.660068 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
Sep 26 13:53:05 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430785, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:53:05.660405 11187 external_mini_cluster.cc:1428] Setting key 56539dc27630d15a424ba4557d4fd0cb
I20260926 13:53:05.665041 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.652222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:47093 (local address 127.10.236.254:44051)
0926 13:53:05.652391 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.652395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.653274 (+ 879us) server_negotiation.cc:407] Connection header received
0926 13:53:05.654618 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.654621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.654684 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.654782 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:05.656063 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.656601 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.657739 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.657967 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.658078 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.660998 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:05.661002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:05.661004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:05.661029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:05.662859 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.663355 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.663358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.663359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.663443 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.663837 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.663840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.663841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.664078 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:05.664160 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.664799 (+ 639us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.664909 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 13:53:05.665882 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.652501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:47093)
0926 13:53:05.653100 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:05.653158 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:05.654392 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:05.654967 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:05.654974 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:05.655326 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:05.655920 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.655934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.656726 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.656731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.657613 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.657622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.658120 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.658123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.658300 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.658937 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:05.658954 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:05.660798 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:05.662960 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:05.662965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:05.662977 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:05.663245 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:05.663610 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:05.663612 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:05.663614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:05.663741 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:05.664215 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:05.664222 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:05.664553 (+ 331us) client_negotiation.cc:769] Sending connection context
0926 13:53:05.664736 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 13:53:05.665015 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":129,"threads_started":1}
I20260926 13:53:05.667029 27057 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44051
I20260926 13:53:05.667277 27057 heartbeater.cc:464] Registering TS with master...
I20260926 13:53:05.667805 27057 heartbeater.cc:511] Master 127.10.236.254:44051 requested a full tablet report, sending...
I20260926 13:53:05.668901 26598 ts_manager.cc:194] Registered new tserver with Master: 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963)
I20260926 13:53:05.669638 26598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:47093
I20260926 13:53:05.674649 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:53:05.684597 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.676802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.10.236.254:44051)
0926 13:53:05.676931 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.676935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.677169 (+ 234us) server_negotiation.cc:407] Connection header received
0926 13:53:05.677277 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.677279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.677318 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.677391 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:05.678194 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.678738 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.679657 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.679867 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.679947 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.681514 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:05.681516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:05.681519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:05.681538 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:05.683007 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.683430 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.683433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.683434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.683608 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:05.683863 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:05.683865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:05.683866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:05.684080 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:05.684181 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.684386 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.684480 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 13:53:05.687446 26598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:53:05.690438 26598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:53:05.705422 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.699446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.10.236.193:35817)
0926 13:53:05.699843 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.699857 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.700112 (+ 255us) server_negotiation.cc:407] Connection header received
0926 13:53:05.700370 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.700375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.700543 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.700706 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:05.701242 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.702525 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.703912 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.704681 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.704817 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.704943 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.705175 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.705273 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":142,"threads_started":1}
I20260926 13:53:05.705478 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.699000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.10.236.194:44475)
0926 13:53:05.699338 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.699352 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.700058 (+ 706us) server_negotiation.cc:407] Connection header received
0926 13:53:05.700406 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.700411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.700563 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.700711 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:05.701246 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.702297 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.703857 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.704450 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.704597 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.704740 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.705186 (+ 446us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.705289 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":151,"threads_started":1}
I20260926 13:53:05.705873 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.698801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44475 (local address 127.0.0.1:47386)
0926 13:53:05.699904 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:05.699956 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:05.700156 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:05.700699 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:05.700708 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:05.700735 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:05.701090 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.701098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.702451 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.702453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.703726 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.703736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.704818 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.704822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.704969 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.705017 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 13:53:05.705640 (+ 623us) client_negotiation.cc:240] Negotiation successful
0926 13:53:05.705718 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":155,"spinlock_wait_cycles":2304,"thread_start_us":78,"threads_started":1}
I20260926 13:53:05.706084 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.699439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:34963 (local address 127.0.0.1:51448)
0926 13:53:05.700940 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:05.700979 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:05.701111 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:05.701584 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:05.701587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:05.701606 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:05.701688 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.701694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.703289 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.703293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.704413 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.704423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.705671 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.705675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.705770 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.705790 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:53:05.705864 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 13:53:05.705941 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1439,"thread_start_us":101,"threads_started":1}
I20260926 13:53:05.706224 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.698730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35817 (local address 127.0.0.1:40790)
0926 13:53:05.699991 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:05.700008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:05.700157 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:05.700697 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:05.700704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:05.700737 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:05.701090 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.701098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.702684 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.702687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.703770 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:05.703779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.704803 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.704806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:05.704911 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.705001 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 13:53:05.705131 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 13:53:05.705257 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":111,"spinlock_wait_cycles":2944,"thread_start_us":78,"threads_started":1}
I20260926 13:53:05.706624 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:05.699859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.10.236.195:34963)
0926 13:53:05.700283 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:05.700295 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:05.701101 (+ 806us) server_negotiation.cc:407] Connection header received
0926 13:53:05.701219 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:05.701225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:05.701428 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:05.701582 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:05.701782 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.703144 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.704548 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:05.705334 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:05.705457 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:05.706335 (+ 878us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:05.706388 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 13:53:05.706497 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":119,"threads_started":1}
I20260926 13:53:05.709450 26991 tablet_service.cc:1467] Processing CreateTablet for tablet b6f60343d26346ed8ce8e3ee8b30a6bd (DEFAULT_TABLE table=test-table [id=8dba99479a60460489c1a8ab0c85b991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:53:05.709399 26723 tablet_service.cc:1467] Processing CreateTablet for tablet b6f60343d26346ed8ce8e3ee8b30a6bd (DEFAULT_TABLE table=test-table [id=8dba99479a60460489c1a8ab0c85b991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:53:05.709401 26857 tablet_service.cc:1467] Processing CreateTablet for tablet b6f60343d26346ed8ce8e3ee8b30a6bd (DEFAULT_TABLE table=test-table [id=8dba99479a60460489c1a8ab0c85b991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:53:05.710484 26991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f60343d26346ed8ce8e3ee8b30a6bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:05.710482 26857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f60343d26346ed8ce8e3ee8b30a6bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:05.710482 26723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f60343d26346ed8ce8e3ee8b30a6bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:05.716598 27074 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Bootstrap starting.
I20260926 13:53:05.718154 27075 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap starting.
I20260926 13:53:05.719136 27074 tablet_bootstrap.cc:654] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:05.719808 27076 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Bootstrap starting.
I20260926 13:53:05.720119 27075 tablet_bootstrap.cc:654] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:05.720593 27074 log.cc:824] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:05.720918 27075 log.cc:824] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:05.722203 27076 tablet_bootstrap.cc:654] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:05.722749 27074 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: No bootstrap required, opened a new log
I20260926 13:53:05.722745 27075 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: No bootstrap required, opened a new log
I20260926 13:53:05.722873 27076 log.cc:824] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:05.722939 27075 ts_tablet_manager.cc:1397] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:53:05.722939 27074 ts_tablet_manager.cc:1397] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 13:53:05.724674 27076 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: No bootstrap required, opened a new log
I20260926 13:53:05.724875 27076 ts_tablet_manager.cc:1397] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:53:05.727100 27074 raft_consensus.cc:348] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:05.727535 27074 raft_consensus.cc:374] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:05.727382 27075 raft_consensus.cc:348] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:05.727603 27074 raft_consensus.cc:729] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3782b56c1afa48da8461fd773ff1b746, State: Initialized, Role: FOLLOWER
I20260926 13:53:05.727710 27075 raft_consensus.cc:374] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:05.727771 27075 raft_consensus.cc:729] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2773e3cc6ad143bd8fae5ac104378c8b, State: Initialized, Role: FOLLOWER
I20260926 13:53:05.728153 27074 consensus_queue.cc:260] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:05.728153 27075 consensus_queue.cc:260] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:05.728178 27076 raft_consensus.cc:348] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:05.728492 27076 raft_consensus.cc:374] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:05.728580 27076 raft_consensus.cc:729] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359713cadb03402a9e96cd6f9416e2ee, State: Initialized, Role: FOLLOWER
I20260926 13:53:05.729030 27076 consensus_queue.cc:260] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:05.729261 27074 ts_tablet_manager.cc:1428] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:53:05.729261 27075 ts_tablet_manager.cc:1428] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260926 13:53:05.729651 27057 heartbeater.cc:503] Master 127.10.236.254:44051 was elected leader, sending a full tablet report...
I20260926 13:53:05.729651 26923 heartbeater.cc:503] Master 127.10.236.254:44051 was elected leader, sending a full tablet report...
I20260926 13:53:05.729650 26789 heartbeater.cc:503] Master 127.10.236.254:44051 was elected leader, sending a full tablet report...
I20260926 13:53:05.731129 27076 ts_tablet_manager.cc:1428] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260926 13:53:05.901192 27058 tablet.cc:2369] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:53:05.907019 26924 tablet.cc:2369] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:53:05.935230 26790 tablet.cc:2369] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:53:06.014315 27080 raft_consensus.cc:482] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:53:06.014587 27080 raft_consensus.cc:504] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:06.015877 27080 leader_election.cc:288] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475), 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963)
I20260926 13:53:06.020188 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.016451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:32941 (local address 127.10.236.194:44475)
0926 13:53:06.016594 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:06.016601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:06.016852 (+ 251us) server_negotiation.cc:407] Connection header received
0926 13:53:06.017032 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:06.017061 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:06.017105 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:06.017197 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:06.017477 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.017994 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.019130 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.019582 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.019708 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:06.019761 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:06.019943 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 13:53:06.020011 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:53:06.020217 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.016320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44475 (local address 127.10.236.193:32941)
0926 13:53:06.016742 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:06.016754 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:06.016904 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:06.017210 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:06.017213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:06.017229 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:06.017361 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:06.017367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.018115 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.018121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:06.018996 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:06.019006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.019706 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.019709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:06.019806 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:06.019829 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:53:06.019927 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 13:53:06.019996 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":25,"thread_start_us":121,"threads_started":1}
I20260926 13:53:06.020360 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.016441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:57041 (local address 127.10.236.195:34963)
0926 13:53:06.016586 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:06.016593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:06.017075 (+ 482us) server_negotiation.cc:407] Connection header received
0926 13:53:06.017244 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:06.017248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:06.017313 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:06.017408 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:06.017718 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.018213 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.019308 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.019739 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.019883 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:06.019941 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:06.020139 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 13:53:06.020210 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:53:06.020380 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.016320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:34963 (local address 127.10.236.193:57041)
0926 13:53:06.016955 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:06.016967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:06.017111 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:06.017418 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:06.017421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:06.017438 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:06.017589 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:06.017595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.018343 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.018349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:06.019155 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:06.019165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.019874 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.019878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:06.019994 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:06.020024 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 13:53:06.020124 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 13:53:06.020195 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":294,"thread_start_us":109,"threads_started":1}
I20260926 13:53:06.021014 27011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f60343d26346ed8ce8e3ee8b30a6bd" candidate_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3782b56c1afa48da8461fd773ff1b746" is_pre_election: true
I20260926 13:53:06.021013 26877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f60343d26346ed8ce8e3ee8b30a6bd" candidate_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359713cadb03402a9e96cd6f9416e2ee" is_pre_election: true
I20260926 13:53:06.021334 27011 raft_consensus.cc:2454] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2773e3cc6ad143bd8fae5ac104378c8b in term 0.
I20260926 13:53:06.021359 26877 raft_consensus.cc:2454] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2773e3cc6ad143bd8fae5ac104378c8b in term 0.
I20260926 13:53:06.021893 26676 leader_election.cc:302] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b, 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746; no voters:
I20260926 13:53:06.022204 27080 raft_consensus.cc:2781] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:53:06.022311 27080 raft_consensus.cc:482] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:53:06.022388 27080 raft_consensus.cc:3037] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:06.023459 27080 raft_consensus.cc:504] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:06.023891 27080 leader_election.cc:288] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 1 election: Requested vote from peers 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475), 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963)
I20260926 13:53:06.024281 26877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f60343d26346ed8ce8e3ee8b30a6bd" candidate_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359713cadb03402a9e96cd6f9416e2ee"
I20260926 13:53:06.024387 27011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f60343d26346ed8ce8e3ee8b30a6bd" candidate_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3782b56c1afa48da8461fd773ff1b746"
I20260926 13:53:06.024405 26877 raft_consensus.cc:3037] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:06.024557 27011 raft_consensus.cc:3037] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:06.025497 26877 raft_consensus.cc:2454] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2773e3cc6ad143bd8fae5ac104378c8b in term 1.
I20260926 13:53:06.025727 27011 raft_consensus.cc:2454] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2773e3cc6ad143bd8fae5ac104378c8b in term 1.
I20260926 13:53:06.025827 26678 leader_election.cc:302] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b, 359713cadb03402a9e96cd6f9416e2ee; no voters:
I20260926 13:53:06.026015 27080 raft_consensus.cc:2781] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:53:06.026265 27080 raft_consensus.cc:686] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 LEADER]: Becoming Leader. State: Replica: 2773e3cc6ad143bd8fae5ac104378c8b, State: Running, Role: LEADER
I20260926 13:53:06.026650 27080 consensus_queue.cc:237] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:06.029963 26596 catalog_manager.cc:5495] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193). New cstate: current_term: 1 leader_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } health_report { overall_health: UNKNOWN } } }
I20260926 13:53:06.068681 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.065034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.10.236.193:35817)
0926 13:53:06.065167 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:06.065170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:06.065184 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:53:06.065262 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:06.065265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:06.065324 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:06.065432 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:53:06.065728 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.066249 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.067070 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:06.067296 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:06.067377 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:06.067756 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:53:06.068186 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:53:06.068296 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:06.068460 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 13:53:06.068542 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:53:06.075120 27011 raft_consensus.cc:1261] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Refusing update from remote peer 2773e3cc6ad143bd8fae5ac104378c8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:53:06.075104 26877 raft_consensus.cc:1261] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Refusing update from remote peer 2773e3cc6ad143bd8fae5ac104378c8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:53:06.075884 27085 consensus_queue.cc:1035] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:53:06.076095 27080 consensus_queue.cc:1035] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:53:06.084578 27090 mvcc.cc:205] Tried to move back new op lower bound from 7333604499755044864 to 7333604499567882240. Current Snapshot: MvccSnapshot[applied={T|T < 7333604499755044864}]
I20260926 13:53:06.085742 27089 mvcc.cc:205] Tried to move back new op lower bound from 7333604499755044864 to 7333604499567882240. Current Snapshot: MvccSnapshot[applied={T|T < 7333604499755044864}]
I20260926 13:53:06.091884 26596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:53:06.092072 26596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:53:06.094501 26596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Sending DeleteTablet for 3 replicas of tablet b6f60343d26346ed8ce8e3ee8b30a6bd
I20260926 13:53:06.095376 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26660
I20260926 13:53:06.095383 26857 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f60343d26346ed8ce8e3ee8b30a6bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:53:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:47386
I20260926 13:53:06.097298 26991 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f60343d26346ed8ce8e3ee8b30a6bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:53:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:51448
I20260926 13:53:06.097406 27097 tablet_replica.cc:330] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: stopping tablet replica
I20260926 13:53:06.097743 27097 raft_consensus.cc:2229] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:53:06.097777 27098 tablet_replica.cc:330] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: stopping tablet replica
I20260926 13:53:06.098068 27098 raft_consensus.cc:2229] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:53:06.098099 27097 raft_consensus.cc:2258] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:53:06.098353 27098 raft_consensus.cc:2258] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:53:06.099906 27097 ts_tablet_manager.cc:1905] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:53:06.099905 27098 ts_tablet_manager.cc:1905] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:53:06.102144 26582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:35817 (error 108)
I20260926 13:53:06.102922 27097 ts_tablet_manager.cc:1918] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:53:06.103133 27097 log.cc:1196] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/b6f60343d26346ed8ce8e3ee8b30a6bd
I20260926 13:53:06.103175 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.102866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35817 (local address 127.0.0.1:40798)
0926 13:53:06.102997 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:06.103072 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:35817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 13:53:06.103222 27098 ts_tablet_manager.cc:1918] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:53:06.103269 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26794
I20260926 13:53:06.103312 27098 log.cc:1196] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/b6f60343d26346ed8ce8e3ee8b30a6bd
I20260926 13:53:06.103636 27097 ts_tablet_manager.cc:1939] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Deleting consensus metadata
I20260926 13:53:06.103688 27098 ts_tablet_manager.cc:1939] T b6f60343d26346ed8ce8e3ee8b30a6bd P 3782b56c1afa48da8461fd773ff1b746: Deleting consensus metadata
W20260926 13:53:06.104760 26582 catalog_manager.cc:4584] TS 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6f60343d26346ed8ce8e3ee8b30a6bd: Network error: Client connection negotiation failed: client connection to 127.10.236.193:35817: connect: Connection refused (error 111)
I20260926 13:53:06.105311 26582 catalog_manager.cc:4841] TS 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963): tablet b6f60343d26346ed8ce8e3ee8b30a6bd (table test-table [id=8dba99479a60460489c1a8ab0c85b991]) successfully deleted
I20260926 13:53:06.109922 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26928
I20260926 13:53:06.110489 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:06.110061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44475 (local address 127.0.0.1:47394)
0926 13:53:06.110278 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:06.110340 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.194:44475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260926 13:53:06.116065 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 26566
I20260926 13:53:26.123106 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44051
--webserver_interface=127.10.236.254
--webserver_port=42955
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:26.242978 27100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:26.243261 27100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:26.243304 27100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:26.247398 27100 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 13:53:26.247462 27100 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 13:53:26.247494 27100 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 13:53:26.247509 27100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:53:26.247653 27100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:26.247712 27100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:26.247731 27100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:53:26.247778 27100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:53:26.253924 27100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=42955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:26.255038 27100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:26.256166 27100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:26.270390 27109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:26.270478 27106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:26.270474 27100 server_base.cc:1034] running on GCE node
W20260926 13:53:26.270390 27107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:26.271380 27100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:26.272697 27100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:26.273907 27100 hybrid_clock.cc:648] HybridClock initialized: now 1790430806273892 us; error 27 us; skew 500 ppm
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:26.282513 27100 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:53:26.283818 27100 webserver.cc:460] Webserver started at http://127.10.236.254:42955/ using document root <none> and password file <none>
I20260926 13:53:26.284485 27100 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:26.284622 27100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:26.289407 27100 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:53:26.292080 27116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:26.293464 27100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:53:26.293684 27100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "a3dbd26d7356439da0bfcc43e68709e5"
format_stamp: "Formatted at 2026-09-26 13:53:04 on dist-test-slave-r34j"
server_key: "4ddbc759f7266dbc384bd8f84804dc7f"
server_key_iv: "2aac666281c1bf9f46f139462acecde6"
server_key_version: "encryptionkey@0"
I20260926 13:53:26.294344 27100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:26.309717 27100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:26.314112 27100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:26.314370 27100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:26.323837 27100 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44051
I20260926 13:53:26.324187 27168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44051 every 8 connection(s)
I20260926 13:53:26.325213 27100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:53:26.330330 27169 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:53:26.331449 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27100
I20260926 13:53:26.331956 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:35817
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:44051
--webserver_port=39555
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:53:26.332561 27169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap starting.
I20260926 13:53:26.351472 27169 log.cc:824] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:26.360671 27169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:53:26.361110 27169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap complete.
I20260926 13:53:26.365335 27169 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:26.365758 27169 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3dbd26d7356439da0bfcc43e68709e5, State: Initialized, Role: FOLLOWER
I20260926 13:53:26.366282 27169 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:26.366398 27169 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:53:26.366492 27169 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:53:26.366616 27169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:53:26.368747 27169 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:26.369127 27169 leader_election.cc:302] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3dbd26d7356439da0bfcc43e68709e5; no voters:
I20260926 13:53:26.369446 27169 leader_election.cc:288] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:53:26.369613 27173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:53:26.369889 27173 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 LEADER]: Becoming Leader. State: Replica: a3dbd26d7356439da0bfcc43e68709e5, State: Running, Role: LEADER
I20260926 13:53:26.370313 27173 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:26.370765 27169 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:53:26.371883 27174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a3dbd26d7356439da0bfcc43e68709e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } } }
I20260926 13:53:26.372014 27174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:53:26.372460 27175 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3dbd26d7356439da0bfcc43e68709e5. Latest consensus state: current_term: 2 leader_uuid: "a3dbd26d7356439da0bfcc43e68709e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } } }
I20260926 13:53:26.372607 27175 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:53:26.372934 27182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:53:26.378732 27182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:53:26.379339 27182 catalog_manager.cc:1259] Loaded cluster ID: 41b1b388b154484fbccf886c866228de
I20260926 13:53:26.379401 27182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:53:26.382087 27182 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:53:26.383430 27182 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Loaded TSK: 0
I20260926 13:53:26.399147 27182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Generated new TSK 1
I20260926 13:53:26.399587 27182 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:53:26.461215 27171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:26.461537 27171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:26.461585 27171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:26.465921 27171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:26.466042 27171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:26.466147 27171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:53:26.472847 27171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:35817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=39555
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:26.474061 27171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:26.475176 27171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:26.497797 27199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:26.497699 27196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:26.497599 27197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:26.498008 27171 server_base.cc:1034] running on GCE node
I20260926 13:53:26.498752 27171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:26.499737 27171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:26.500983 27171 hybrid_clock.cc:648] HybridClock initialized: now 1790430806500956 us; error 39 us; skew 500 ppm
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:26.504839 27171 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:53:26.506093 27171 webserver.cc:460] Webserver started at http://127.10.236.193:39555/ using document root <none> and password file <none>
I20260926 13:53:26.506939 27171 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:26.507081 27171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:26.512271 27171 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:53:26.515035 27206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:26.517208 27171 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:53:26.517354 27171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2773e3cc6ad143bd8fae5ac104378c8b"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "01a9895ba57a65e9a8e57b42aa91cfeb"
server_key_iv: "75121fa2c3d5831de6ac26c44ea804f7"
server_key_version: "encryptionkey@0"
I20260926 13:53:26.518152 27171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:26.539395 27171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:26.543284 27171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:26.543569 27171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:26.544229 27171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:26.545493 27213 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:53:26.550158 27171 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:53:26.550259 27171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260926 13:53:26.550344 27171 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:53:26.552186 27171 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:53:26.552256 27171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260926 13:53:26.552630 27213 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap starting.
I20260926 13:53:26.564074 27171 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:35817
I20260926 13:53:26.564121 27320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:35817 every 8 connection(s)
I20260926 13:53:26.565215 27171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:53:26.571276 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27171
I20260926 13:53:26.571931 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:44475
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:44051
--webserver_port=42661
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:53:26.574041 27213 log.cc:824] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:53:26.586479 27213 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:53:26.586786 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:26.567715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:36821)
0926 13:53:26.568246 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:26.568290 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:26.569138 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:26.570359 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:26.570368 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:26.570985 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:26.571733 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:26.571780 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.573473 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.573478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:26.574956 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:26.574968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.576480 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.576486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:26.576801 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:26.577564 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:26.577581 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:26.579127 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:26.582037 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:26.582045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:26.582061 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:26.582485 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:26.583137 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:26.583144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:26.583148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:26.583370 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:26.584404 (+ 1034us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:26.584412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:26.584801 (+ 389us) client_negotiation.cc:769] Sending connection context
0926 13:53:26.585054 (+ 253us) client_negotiation.cc:240] Negotiation successful
0926 13:53:26.585453 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":150,"threads_started":1}
I20260926 13:53:26.587795 27213 tablet_bootstrap.cc:492] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap complete.
I20260926 13:53:26.589924 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:26.567636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:36821 (local address 127.10.236.254:44051)
0926 13:53:26.568058 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:26.568079 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:26.568416 (+ 337us) server_negotiation.cc:407] Connection header received
0926 13:53:26.569608 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:26.569618 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:26.569944 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:26.570298 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:26.571942 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.573247 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.575095 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.576291 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.576524 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:26.579374 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:26.579380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:26.579394 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:26.579429 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:26.581865 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:26.582651 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:26.582660 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:26.582668 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:26.582976 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:26.583542 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:26.583546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:26.583548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:26.584163 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:26.584330 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:26.588300 (+ 3970us) server_negotiation.cc:299] Negotiation successful
0926 13:53:26.588731 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":124,"threads_started":1}
I20260926 13:53:26.591594 27213 ts_tablet_manager.cc:1397] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent bootstrapping tablet: real 0.039s user 0.028s sys 0.008s
I20260926 13:53:26.593647 27321 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44051
I20260926 13:53:26.593920 27321 heartbeater.cc:464] Registering TS with master...
I20260926 13:53:26.594527 27321 heartbeater.cc:511] Master 127.10.236.254:44051 requested a full tablet report, sending...
I20260926 13:53:26.594982 27213 raft_consensus.cc:348] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:26.595477 27213 raft_consensus.cc:729] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2773e3cc6ad143bd8fae5ac104378c8b, State: Initialized, Role: FOLLOWER
I20260926 13:53:26.596091 27213 consensus_queue.cc:260] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } }
I20260926 13:53:26.597281 27213 ts_tablet_manager.cc:1428] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260926 13:53:26.597776 27133 ts_manager.cc:194] Registered new tserver with Master: 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817)
I20260926 13:53:26.602105 27133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:36821
I20260926 13:53:26.605263 27321 heartbeater.cc:503] Master 127.10.236.254:44051 was elected leader, sending a full tablet report...
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.193@KRBTEST.COM
I20260926 13:53:26.613606 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:26.601164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35817 (local address 127.0.0.1:40830)
0926 13:53:26.601621 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:26.601671 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:26.601812 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:26.602903 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:26.602912 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:26.603399 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:26.603727 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:26.603741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.604884 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.604890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:26.605810 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:26.605825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.606660 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.606664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:26.606780 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:26.607553 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:26.607580 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:26.609189 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:26.611618 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:26.611630 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:26.611650 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:26.611929 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:26.612270 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:26.612273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:26.612275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:26.612367 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:26.612874 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:26.612880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:26.613018 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 13:53:26.613168 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 13:53:26.613396 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":131,"threads_started":1}
I20260926 13:53:26.613692 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:26.601241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.10.236.193:35817)
0926 13:53:26.602057 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:26.602070 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:26.602101 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:53:26.602332 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:26.602348 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:26.602592 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:26.602890 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:26.603888 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.604737 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.605939 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.606516 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.606650 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:26.609363 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:26.609383 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:26.609396 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:26.609423 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:26.611412 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:26.612061 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:26.612069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:26.612076 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:26.612164 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:26.612476 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:26.612480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:26.612481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:26.612750 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:26.612899 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:26.613260 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 13:53:26.613430 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":698,"thread_start_us":138,"threads_started":1}
I20260926 13:53:26.614989 27255 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f60343d26346ed8ce8e3ee8b30a6bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:53:06 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:40830
I20260926 13:53:26.616345 27330 tablet_replica.cc:330] stopping tablet replica
I20260926 13:53:26.616678 27330 raft_consensus.cc:2229] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:53:26.616954 27330 raft_consensus.cc:2258] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:53:26.618003 27330 ts_tablet_manager.cc:1905] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:53:26.622803 27330 ts_tablet_manager.cc:1918] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:53:26.623128 27330 log.cc:1196] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal/wals/b6f60343d26346ed8ce8e3ee8b30a6bd
I20260926 13:53:26.623716 27330 ts_tablet_manager.cc:1939] T b6f60343d26346ed8ce8e3ee8b30a6bd P 2773e3cc6ad143bd8fae5ac104378c8b: Deleting consensus metadata
I20260926 13:53:26.625898 27117 catalog_manager.cc:4841] TS 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817): tablet b6f60343d26346ed8ce8e3ee8b30a6bd (table test-table [id=8dba99479a60460489c1a8ab0c85b991]) successfully deleted
W20260926 13:53:26.708240 27325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:26.708554 27325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:26.708604 27325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:26.713402 27325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:26.713482 27325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:26.713551 27325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:53:26.719781 27325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:44475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=42661
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:26.720925 27325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:26.721993 27325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:26.732288 27338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:26.732228 27336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:26.732362 27325 server_base.cc:1034] running on GCE node
W20260926 13:53:26.732273 27335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:26.732983 27325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:26.733654 27325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:26.734936 27325 hybrid_clock.cc:648] HybridClock initialized: now 1790430806734904 us; error 43 us; skew 500 ppm
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:26.738091 27325 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:53:26.739077 27325 webserver.cc:460] Webserver started at http://127.10.236.194:42661/ using document root <none> and password file <none>
I20260926 13:53:26.739770 27325 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:26.739857 27325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:26.744148 27325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:53:26.746474 27345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:26.747985 27325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:53:26.748096 27325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "359713cadb03402a9e96cd6f9416e2ee"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "8d00e1ca0d1f2e8ccff4ae3e97deda5f"
server_key_iv: "615100bdf380ab5969a430b2ee8c9f2a"
server_key_version: "encryptionkey@0"
I20260926 13:53:26.748656 27325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:26.773640 27325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:26.777623 27325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:26.777813 27325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:26.778402 27325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:26.779688 27352 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 13:53:26.783689 27352 ts_tablet_manager.cc:1852] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 13:53:26.784094 27352 ts_tablet_manager.cc:1905] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:53:26.788121 27352 ts_tablet_manager.cc:1918] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:53:26.788285 27352 ts_tablet_manager.cc:1939] T b6f60343d26346ed8ce8e3ee8b30a6bd P 359713cadb03402a9e96cd6f9416e2ee: Deleting consensus metadata
I20260926 13:53:26.789175 27325 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 13:53:26.789286 27325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.001s
I20260926 13:53:26.789407 27325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:26.789460 27325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:26.798996 27325 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:44475
I20260926 13:53:26.799137 27459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:44475 every 8 connection(s)
I20260926 13:53:26.800009 27325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:53:26.805265 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27325
I20260926 13:53:26.805749 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:34963
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:44051
--webserver_port=42827
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:53:26.815213 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:26.801957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:53823 (local address 127.10.236.254:44051)
0926 13:53:26.802122 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:26.802127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:26.803090 (+ 963us) server_negotiation.cc:407] Connection header received
0926 13:53:26.804195 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:26.804199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:26.804260 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:26.804346 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:26.805758 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.806399 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.808085 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.808361 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.808465 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:26.811561 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:26.811564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:26.811566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:26.811594 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:26.813092 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:26.813652 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:26.813662 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:26.813664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:26.813740 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:26.814089 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:26.814092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:26.814094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:26.814289 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:26.814395 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:26.814902 (+ 507us) server_negotiation.cc:299] Negotiation successful
0926 13:53:26.815044 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 13:53:26.816241 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:26.802426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:53823)
0926 13:53:26.802918 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:26.802967 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:26.803980 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:26.804552 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:26.804559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:26.804918 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:26.805544 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:26.805561 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.806534 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.806540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:26.807931 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:26.807945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:26.808535 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:26.808541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:26.808782 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:26.809443 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:26.809469 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:26.811328 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:26.813224 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:26.813230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:26.813248 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:26.813537 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:26.813829 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:26.813833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:26.813835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:26.813985 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:26.814392 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:26.814399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:26.814665 (+ 266us) client_negotiation.cc:769] Sending connection context
0926 13:53:26.814862 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 13:53:26.815209 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":113,"threads_started":1}
I20260926 13:53:26.817780 27460 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44051
I20260926 13:53:26.818048 27460 heartbeater.cc:464] Registering TS with master...
I20260926 13:53:26.818570 27460 heartbeater.cc:511] Master 127.10.236.254:44051 requested a full tablet report, sending...
I20260926 13:53:26.819783 27133 ts_manager.cc:194] Registered new tserver with Master: 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
I20260926 13:53:26.820487 27133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:53823
W20260926 13:53:26.926977 27463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:26.927316 27463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:26.927393 27463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:26.931700 27463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:26.931798 27463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:26.931895 27463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:53:26.937415 27463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:34963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=42827
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:26.938524 27463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:26.939594 27463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:26.950048 27469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:26.950060 27471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:26.950021 27468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:26.950450 27463 server_base.cc:1034] running on GCE node
I20260926 13:53:26.951073 27463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:26.951906 27463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:26.953118 27463 hybrid_clock.cc:648] HybridClock initialized: now 1790430806953078 us; error 53 us; skew 500 ppm
Sep 26 13:53:26 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:26.956311 27463 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:53:26.957306 27463 webserver.cc:460] Webserver started at http://127.10.236.195:42827/ using document root <none> and password file <none>
I20260926 13:53:26.957958 27463 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:26.958082 27463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:26.962385 27463 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:53:26.964959 27478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:26.966578 27463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:53:26.966776 27463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "3782b56c1afa48da8461fd773ff1b746"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "7c79b7e85c1afb7068618e7f5765fae1"
server_key_iv: "fbb69a86152f18f50609eafbf0e31fc6"
server_key_version: "encryptionkey@0"
I20260926 13:53:26.967413 27463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:27.000946 27463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:27.004088 27463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:27.004315 27463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:27.004918 27463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:27.005764 27463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:53:27.005844 27463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:27.005940 27463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:27.005992 27463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:27.015661 27463 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:34963
I20260926 13:53:27.015700 27591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:34963 every 8 connection(s)
I20260926 13:53:27.016716 27463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:53:27.024000 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27463
Sep 26 13:53:27 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430806, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:53:27.031929 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.018646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:56443 (local address 127.10.236.254:44051)
0926 13:53:27.018853 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.018858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.019746 (+ 888us) server_negotiation.cc:407] Connection header received
0926 13:53:27.021033 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.021036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.021088 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.021173 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:27.022515 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.023063 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.024499 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.024937 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.025045 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.027823 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:27.027827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:27.027830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:27.027859 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:27.029473 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:27.030145 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:27.030148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:27.030150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:27.030303 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:27.030716 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:27.030719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:27.030721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:27.030908 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:27.031018 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.031567 (+ 549us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.031751 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 13:53:27.032732 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.019069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:56443)
0926 13:53:27.019590 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:27.019633 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:27.020810 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:27.021398 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:27.021406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:27.021833 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:27.022344 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.022359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.023210 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.023213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.024351 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.024361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.025087 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.025091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.025271 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.025911 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:27.025928 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:27.027644 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:27.029622 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:27.029634 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:27.029655 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:27.030036 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:27.030398 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:27.030400 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:27.030401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:27.030544 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:27.031015 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:27.031023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:27.031299 (+ 276us) client_negotiation.cc:769] Sending connection context
0926 13:53:27.031492 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 13:53:27.031887 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":127,"threads_started":1}
I20260926 13:53:27.033871 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.024716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.10.236.254:44051)
0926 13:53:27.024942 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.024946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.025107 (+ 161us) server_negotiation.cc:407] Connection header received
0926 13:53:27.025268 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.025271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.025324 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.025426 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:27.026177 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.026899 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.027999 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.028260 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.028402 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.030161 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:27.030164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:27.030167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:27.030189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:27.031935 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:27.032558 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:27.032563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:27.032565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:27.032647 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:27.032965 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:27.032969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:27.032970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:27.033154 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:27.033375 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.033551 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.033681 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260926 13:53:27.034029 27592 heartbeater.cc:347] Connected to a master server at 127.10.236.254:44051
I20260926 13:53:27.034350 27592 heartbeater.cc:464] Registering TS with master...
I20260926 13:53:27.034929 27592 heartbeater.cc:511] Master 127.10.236.254:44051 requested a full tablet report, sending...
I20260926 13:53:27.035935 27133 ts_manager.cc:194] Registered new tserver with Master: 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963)
I20260926 13:53:27.036489 27133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:56443
I20260926 13:53:27.048141 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:53:27.052879 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.048880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.10.236.254:44051)
0926 13:53:27.049097 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.049102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.049211 (+ 109us) server_negotiation.cc:407] Connection header received
0926 13:53:27.049440 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.049442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.049552 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.049672 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:53:27.049923 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.050448 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.051536 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.051780 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.051897 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.052266 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:53:27.052415 (+ 149us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 13:53:27.052711 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44832: token expired
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 13:53:27.052977 27595 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44832: token expired
W20260926 13:53:27.053396 27060 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.236.254:44051)
I20260926 13:53:27.062204 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.053807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.10.236.254:44051)
0926 13:53:27.053931 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.053934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.053952 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:53:27.054130 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.054133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.054174 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.054245 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:27.055124 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.055808 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.056649 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.056857 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.056962 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.058627 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:27.058630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:27.058632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:27.058657 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:27.060155 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:27.060729 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:27.060732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:27.060734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:27.060822 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:27.061129 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:27.061132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:27.061133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:27.061316 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:27.061440 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.061855 (+ 415us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.061962 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:53:27.065069 27133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:53:27.068898 27133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:53:27.077801 27255 tablet_service.cc:1467] Processing CreateTablet for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 (DEFAULT_TABLE table=test-table [id=7e5b19c93048474b839e210ba9a702fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:53:27.078543 27255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:27.081888 27213 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap starting.
I20260926 13:53:27.082592 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.076686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:34963 (local address 127.0.0.1:53636)
0926 13:53:27.076827 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:27.076846 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:27.076935 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:27.077998 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:27.078001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:27.078022 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:27.078134 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.078140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.079976 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.079981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.080997 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.081007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.082129 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.082132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.082224 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.082267 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 13:53:27.082367 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 13:53:27.082459 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 13:53:27.082859 27213 tablet_bootstrap.cc:654] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:27.083117 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.077036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.10.236.195:34963)
0926 13:53:27.077470 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.077493 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.077521 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:53:27.077619 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.077625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.077839 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.077969 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:27.078384 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.079862 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.081138 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.082005 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.082148 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.082289 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.082880 (+ 591us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.082989 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":85,"threads_started":1}
I20260926 13:53:27.084183 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.076938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44475 (local address 127.0.0.1:49986)
0926 13:53:27.077473 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:27.077488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:27.077580 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:27.078962 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:27.078965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:27.078983 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:27.079065 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.079071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.080565 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.080568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.081864 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.081873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.083673 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.083678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.083812 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.083843 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 13:53:27.083965 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 13:53:27.084043 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":150,"threads_started":1}
I20260926 13:53:27.084262 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.077429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.10.236.194:44475)
0926 13:53:27.078312 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.078324 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.078349 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:53:27.078420 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.078426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.078630 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.078777 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:27.079198 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.080433 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.082072 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.083277 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.083431 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.083600 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.083997 (+ 397us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.084090 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":100,"threads_started":1}
I20260926 13:53:27.085254 27213 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: No bootstrap required, opened a new log
I20260926 13:53:27.085345 27213 ts_tablet_manager.cc:1397] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 13:53:27.085796 27213 raft_consensus.cc:348] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.085908 27213 raft_consensus.cc:374] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:27.085944 27213 raft_consensus.cc:729] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2773e3cc6ad143bd8fae5ac104378c8b, State: Initialized, Role: FOLLOWER
I20260926 13:53:27.086069 27213 consensus_queue.cc:260] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.086345 27213 ts_tablet_manager.cc:1428] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:53:27.085853 27394 tablet_service.cc:1467] Processing CreateTablet for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 (DEFAULT_TABLE table=test-table [id=7e5b19c93048474b839e210ba9a702fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:53:27.086277 27526 tablet_service.cc:1467] Processing CreateTablet for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 (DEFAULT_TABLE table=test-table [id=7e5b19c93048474b839e210ba9a702fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:53:27.086993 27394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:27.087291 27526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:53:27.091871 27352 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Bootstrap starting.
I20260926 13:53:27.094266 27605 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Bootstrap starting.
I20260926 13:53:27.094395 27352 tablet_bootstrap.cc:654] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:27.095357 27352 log.cc:824] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:27.097106 27605 tablet_bootstrap.cc:654] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Neither blocks nor log segments found. Creating new log.
I20260926 13:53:27.097712 27352 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: No bootstrap required, opened a new log
I20260926 13:53:27.097916 27352 ts_tablet_manager.cc:1397] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:53:27.098021 27605 log.cc:824] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:27.100332 27605 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: No bootstrap required, opened a new log
I20260926 13:53:27.100538 27605 ts_tablet_manager.cc:1397] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 13:53:27.102384 27352 raft_consensus.cc:348] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.102681 27352 raft_consensus.cc:374] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:27.102737 27352 raft_consensus.cc:729] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359713cadb03402a9e96cd6f9416e2ee, State: Initialized, Role: FOLLOWER
I20260926 13:53:27.103252 27352 consensus_queue.cc:260] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.104048 27460 heartbeater.cc:503] Master 127.10.236.254:44051 was elected leader, sending a full tablet report...
I20260926 13:53:27.104485 27352 ts_tablet_manager.cc:1428] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:53:27.104843 27605 raft_consensus.cc:348] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.105129 27605 raft_consensus.cc:374] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:53:27.105201 27605 raft_consensus.cc:729] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3782b56c1afa48da8461fd773ff1b746, State: Initialized, Role: FOLLOWER
I20260926 13:53:27.105644 27605 consensus_queue.cc:260] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.106516 27592 heartbeater.cc:503] Master 127.10.236.254:44051 was elected leader, sending a full tablet report...
I20260926 13:53:27.106735 27605 ts_tablet_manager.cc:1428] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 13:53:27.199385 27609 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:53:27.199739 27609 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.201278 27609 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475), 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817)
I20260926 13:53:27.205785 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.201762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:53249 (local address 127.10.236.193:35817)
0926 13:53:27.202244 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.202249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.202268 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:53:27.202335 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.202337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.202398 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.202534 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:27.202860 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.203401 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.204553 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.205111 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.205283 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.205472 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.205569 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.205646 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":121,"threads_started":1}
I20260926 13:53:27.205791 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.201604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35817 (local address 127.10.236.195:53249)
0926 13:53:27.201811 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:27.201830 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:27.202023 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:27.202568 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:27.202571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:27.202591 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:27.202729 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.202735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.203565 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.203569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.204391 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.204400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.205282 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.205286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.205435 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.205457 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:53:27.205545 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:53:27.205628 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":33}
I20260926 13:53:27.205899 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.201851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:37691 (local address 127.10.236.194:44475)
0926 13:53:27.202056 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.202063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.202464 (+ 401us) server_negotiation.cc:407] Connection header received
0926 13:53:27.202628 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.202630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.202676 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.202756 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:53:27.203024 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.203597 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.204739 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.205231 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.205376 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.205442 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.205670 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.205749 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 13:53:27.206172 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.201611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44475 (local address 127.10.236.195:37691)
0926 13:53:27.202346 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:27.202358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:27.202484 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:27.202776 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:27.202779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:27.202794 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:53:27.202909 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.202914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.203757 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.203763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.204524 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:27.204532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.205376 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.205380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:27.205520 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.205540 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:53:27.205638 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 13:53:27.205713 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":356,"thread_start_us":148,"threads_started":1}
I20260926 13:53:27.206465 27275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12ec96ea554ee3bfd2b4a33fa9b5a8" candidate_uuid: "3782b56c1afa48da8461fd773ff1b746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" is_pre_election: true
I20260926 13:53:27.206854 27275 raft_consensus.cc:2454] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3782b56c1afa48da8461fd773ff1b746 in term 0.
I20260926 13:53:27.207041 27414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12ec96ea554ee3bfd2b4a33fa9b5a8" candidate_uuid: "3782b56c1afa48da8461fd773ff1b746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359713cadb03402a9e96cd6f9416e2ee" is_pre_election: true
I20260926 13:53:27.207350 27414 raft_consensus.cc:2454] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3782b56c1afa48da8461fd773ff1b746 in term 0.
I20260926 13:53:27.207441 27479 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b, 3782b56c1afa48da8461fd773ff1b746; no voters:
I20260926 13:53:27.207828 27609 raft_consensus.cc:2781] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:53:27.207934 27609 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:53:27.207985 27609 raft_consensus.cc:3037] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:27.209023 27609 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.209408 27609 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [CANDIDATE]: Term 1 election: Requested vote from peers 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475), 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817)
I20260926 13:53:27.209831 27414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12ec96ea554ee3bfd2b4a33fa9b5a8" candidate_uuid: "3782b56c1afa48da8461fd773ff1b746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359713cadb03402a9e96cd6f9416e2ee"
I20260926 13:53:27.209988 27414 raft_consensus.cc:3037] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:27.210011 27275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12ec96ea554ee3bfd2b4a33fa9b5a8" candidate_uuid: "3782b56c1afa48da8461fd773ff1b746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2773e3cc6ad143bd8fae5ac104378c8b"
I20260926 13:53:27.210188 27275 raft_consensus.cc:3037] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:53:27.211141 27414 raft_consensus.cc:2454] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3782b56c1afa48da8461fd773ff1b746 in term 1.
I20260926 13:53:27.211266 27275 raft_consensus.cc:2454] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3782b56c1afa48da8461fd773ff1b746 in term 1.
I20260926 13:53:27.211516 27481 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746; no voters:
I20260926 13:53:27.211681 27609 raft_consensus.cc:2781] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:53:27.211894 27609 raft_consensus.cc:686] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 1 LEADER]: Becoming Leader. State: Replica: 3782b56c1afa48da8461fd773ff1b746, State: Running, Role: LEADER
I20260926 13:53:27.212273 27609 consensus_queue.cc:237] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:27.215525 27132 catalog_manager.cc:5495] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "3782b56c1afa48da8461fd773ff1b746" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260926 13:53:27.239441 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:27.235930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.10.236.195:34963)
0926 13:53:27.236057 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:27.236061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:27.236074 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:53:27.236149 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:27.236152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:27.236209 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:27.236302 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:53:27.236515 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.237047 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.237980 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:27.238215 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:27.238309 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:27.238621 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:53:27.239059 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:53:27.239187 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:27.239276 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 13:53:27.239330 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:53:27.245875 27275 raft_consensus.cc:1261] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Refusing update from remote peer 3782b56c1afa48da8461fd773ff1b746: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:53:27.245982 27414 raft_consensus.cc:1261] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Refusing update from remote peer 3782b56c1afa48da8461fd773ff1b746: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:53:27.246766 27612 consensus_queue.cc:1035] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [LEADER]: Connected to new peer: Peer: permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:53:27.246977 27609 consensus_queue.cc:1035] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:53:27.252209 27615 mvcc.cc:205] Tried to move back new op lower bound from 7333604586471202816 to 7333604586344337408. Current Snapshot: MvccSnapshot[applied={T|T < 7333604586471202816}]
I20260926 13:53:27.253698 27616 mvcc.cc:205] Tried to move back new op lower bound from 7333604586471202816 to 7333604586344337408. Current Snapshot: MvccSnapshot[applied={T|T < 7333604586471202816}]
I20260926 13:53:27.258867 27620 mvcc.cc:205] Tried to move back new op lower bound from 7333604586471202816 to 7333604586344337408. Current Snapshot: MvccSnapshot[applied={T|T < 7333604586471202816}]
I20260926 13:53:27.262334 27133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:53:27.262529 27133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:53:27.265025 27133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Sending DeleteTablet for 3 replicas of tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:27.265828 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 27171
I20260926 13:53:27.265832 27255 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:53:27 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:40830
I20260926 13:53:27.265806 27394 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:53:27 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:49986
I20260926 13:53:27.266048 27526 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:53:27 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:53636
I20260926 13:53:27.266752 27624 tablet_replica.cc:330] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: stopping tablet replica
I20260926 13:53:27.267096 27624 raft_consensus.cc:2229] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:53:27.267428 27624 raft_consensus.cc:2258] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:53:27.269580 27624 ts_tablet_manager.cc:1905] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:53:27.271601 27625 tablet_replica.cc:330] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: stopping tablet replica
I20260926 13:53:27.272017 27625 raft_consensus.cc:2229] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:53:27.272419 27625 raft_consensus.cc:2258] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:53:27.272677 27624 ts_tablet_manager.cc:1918] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:53:27.272769 27624 log.cc:1196] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:27.273109 27624 ts_tablet_manager.cc:1939] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 359713cadb03402a9e96cd6f9416e2ee: Deleting consensus metadata
W20260926 13:53:27.273851 27117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:35817 (error 108)
I20260926 13:53:27.274386 27119 catalog_manager.cc:4841] TS 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475): tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8 (table test-table [id=7e5b19c93048474b839e210ba9a702fe]) successfully deleted
W20260926 13:53:27.275190 27601 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:53:27.274869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:35817 (local address 127.0.0.1:40846)
0926 13:53:27.274991 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:27.275008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:27.275086 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:35817: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 13:53:27.275409 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 27325
W20260926 13:53:27.275380 27117 catalog_manager.cc:4584] TS 2773e3cc6ad143bd8fae5ac104378c8b (127.10.236.193:35817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e12ec96ea554ee3bfd2b4a33fa9b5a8: Network error: Client connection negotiation failed: client connection to 127.10.236.193:35817: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260926 13:53:27.280640 27625 ts_tablet_manager.cc:1905] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:53:27.283197 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 27463
I20260926 13:53:27.289304 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 27100
I20260926 13:53:47.296248 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44051
--webserver_interface=127.10.236.254
--webserver_port=42955
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:47.409883 27627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:47.410231 27627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:47.410305 27627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:47.414628 27627 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 13:53:47.414716 27627 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 13:53:47.414743 27627 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 13:53:47.414775 27627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:53:47.414817 27627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:47.414858 27627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:47.414896 27627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:53:47.414933 27627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:53:47.420420 27627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=42955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:47.421608 27627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:47.422688 27627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:47.432149 27635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:47.432137 27632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:47.432204 27627 server_base.cc:1034] running on GCE node
W20260926 13:53:47.432153 27633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:47.433197 27627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:47.434450 27627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:47.435591 27627 hybrid_clock.cc:648] HybridClock initialized: now 1790430827435481 us; error 128 us; skew 500 ppm
Sep 26 13:53:47 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430827, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:47.445626 27627 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:53:47.446704 27627 webserver.cc:460] Webserver started at http://127.10.236.254:42955/ using document root <none> and password file <none>
I20260926 13:53:47.447266 27627 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:47.447330 27627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:47.451421 27627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:53:47.454073 27642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:47.455672 27627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 13:53:47.455807 27627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "a3dbd26d7356439da0bfcc43e68709e5"
format_stamp: "Formatted at 2026-09-26 13:53:04 on dist-test-slave-r34j"
server_key: "4ddbc759f7266dbc384bd8f84804dc7f"
server_key_iv: "2aac666281c1bf9f46f139462acecde6"
server_key_version: "encryptionkey@0"
I20260926 13:53:47.456420 27627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:47.477746 27627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:47.481413 27627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:47.481616 27627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:47.490213 27627 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44051
I20260926 13:53:47.490573 27694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44051 every 8 connection(s)
I20260926 13:53:47.491417 27627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:53:47.494006 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27627
I20260926 13:53:47.494510 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:35817
--local_ip_for_outbound_sockets=127.10.236.193
--tserver_master_addrs=127.10.236.254:44051
--webserver_port=39555
--webserver_interface=127.10.236.193
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:53:47.496564 27695 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:53:47.498383 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap starting.
I20260926 13:53:47.516916 27695 log.cc:824] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:47.529718 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:53:47.530169 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Bootstrap complete.
I20260926 13:53:47.534608 27695 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:47.535032 27695 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3dbd26d7356439da0bfcc43e68709e5, State: Initialized, Role: FOLLOWER
I20260926 13:53:47.535625 27695 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:47.535768 27695 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:53:47.535867 27695 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:53:47.535980 27695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 2 FOLLOWER]: Advancing to term 3
I20260926 13:53:47.538115 27695 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:47.538506 27695 leader_election.cc:302] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3dbd26d7356439da0bfcc43e68709e5; no voters:
I20260926 13:53:47.538879 27695 leader_election.cc:288] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 13:53:47.539004 27699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 13:53:47.539243 27699 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [term 3 LEADER]: Becoming Leader. State: Replica: a3dbd26d7356439da0bfcc43e68709e5, State: Running, Role: LEADER
I20260926 13:53:47.539680 27699 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } }
I20260926 13:53:47.540395 27695 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:53:47.541191 27700 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3dbd26d7356439da0bfcc43e68709e5. Latest consensus state: current_term: 3 leader_uuid: "a3dbd26d7356439da0bfcc43e68709e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } } }
I20260926 13:53:47.541338 27700 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:53:47.542191 27701 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a3dbd26d7356439da0bfcc43e68709e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3dbd26d7356439da0bfcc43e68709e5" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44051 } } }
I20260926 13:53:47.542382 27701 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:53:47.542692 27706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:53:47.547216 27706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:53:47.547921 27706 catalog_manager.cc:1259] Loaded cluster ID: 41b1b388b154484fbccf886c866228de
I20260926 13:53:47.548013 27706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:53:47.549973 27706 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:53:47.550633 27706 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Loaded TSK: 0
I20260926 13:53:47.550738 27706 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a3dbd26d7356439da0bfcc43e68709e5: Loaded TSK: 1
F20260926 13:53:47.551390 27706 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8d9986d9ad google::LogMessage::Fail() at ??:0
@ 0x7f8d9987088a google::LogMessage::SendToLog() at ??:0
@ 0x7f8d9986d4de google::LogMessage::Flush() at ??:0
@ 0x7f8d9986ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8d9c53118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8d9c529f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8da41d67bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8da41b099c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8da41b1178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f8da41f2df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f8da421182b std::function<>::operator()() at ??:0
@ 0x7f8da41b0b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8da41b1f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8da41ae579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8da41f2640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d9babf3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8d9babfc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8d9bac13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d9bab0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8d9d0f46db start_thread at ??:0
@ 0x7f8d9836b61f clone at ??:0
W20260926 13:53:47.647397 27697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:47.647804 27697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:47.647955 27697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:47.654055 27697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:47.654189 27697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:47.654328 27697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:53:47.663048 27697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:35817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=39555
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:47.664495 27697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:47.665640 27697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:47.679376 27742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:47.679361 27743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:47.679741 27745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:47.679944 27697 server_base.cc:1034] running on GCE node
I20260926 13:53:47.680500 27697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:47.681315 27697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:47.682528 27697 hybrid_clock.cc:648] HybridClock initialized: now 1790430827682485 us; error 57 us; skew 500 ppm
Sep 26 13:53:47 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430827, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:47.686213 27697 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:53:47.687645 27697 webserver.cc:460] Webserver started at http://127.10.236.193:39555/ using document root <none> and password file <none>
I20260926 13:53:47.688477 27697 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:47.688568 27697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:47.694905 27697 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:53:47.698464 27752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:47.701078 27697 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:53:47.701247 27697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2773e3cc6ad143bd8fae5ac104378c8b"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "01a9895ba57a65e9a8e57b42aa91cfeb"
server_key_iv: "75121fa2c3d5831de6ac26c44ea804f7"
server_key_version: "encryptionkey@0"
I20260926 13:53:47.702230 27697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:47.718262 27697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:47.721805 27697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:47.722023 27697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:47.723078 27697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:47.724434 27759 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:53:47.728817 27697 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:53:47.728889 27697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260926 13:53:47.728958 27697 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:53:47.730614 27697 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:53:47.730674 27697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 13:53:47.730930 27759 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap starting.
I20260926 13:53:47.741588 27697 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:35817
I20260926 13:53:47.741971 27866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:35817 every 8 connection(s)
I20260926 13:53:47.742925 27697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:53:47.751416 27759 log.cc:824] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Log is configured to *not* fsync() on all Append() calls
I20260926 13:53:47.752493 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27697
I20260926 13:53:47.753024 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:44475
--local_ip_for_outbound_sockets=127.10.236.194
--tserver_master_addrs=127.10.236.254:44051
--webserver_port=42661
--webserver_interface=127.10.236.194
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:53:47.759730 27759 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:53:47.760114 27759 tablet_bootstrap.cc:492] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Bootstrap complete.
I20260926 13:53:47.761257 27759 ts_tablet_manager.cc:1397] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.006s
I20260926 13:53:47.763780 27759 raft_consensus.cc:348] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:47.764228 27759 raft_consensus.cc:729] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2773e3cc6ad143bd8fae5ac104378c8b, State: Initialized, Role: FOLLOWER
I20260926 13:53:47.764796 27759 consensus_queue.cc:260] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:47.765763 27759 ts_tablet_manager.cc:1428] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b: Time spent starting tablet: real 0.004s user 0.005s sys 0.001s
W20260926 13:53:47.825829 27869 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:53:47.745326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:33699)
0926 13:53:47.745965 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:47.746025 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:47.747111 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:47.825063 (+ 77952us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":188,"threads_started":1}
W20260926 13:53:47.826493 27867 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:44051 (0 consecutive failures): Network error: Failed to ping master at 127.10.236.254:44051: Client connection negotiation failed: client connection to 127.10.236.254:44051: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:53:47.827272 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:47.827003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:40491)
0926 13:53:47.827134 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:47.827205 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 13:53:47.828037 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:47.827817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:44409)
0926 13:53:47.827935 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:47.827980 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:53:47.828239 27867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:53:47.879318 27871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:47.879634 27871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:47.879701 27871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:47.884492 27871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:47.884572 27871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:47.884636 27871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:53:47.890795 27871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:44475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=42661
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:47.892028 27871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:47.893178 27871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:47.903537 27880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:47.903566 27877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:47.903685 27871 server_base.cc:1034] running on GCE node
W20260926 13:53:47.903569 27878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:47.904312 27871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:47.904919 27871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:47.906138 27871 hybrid_clock.cc:648] HybridClock initialized: now 1790430827906101 us; error 48 us; skew 500 ppm
Sep 26 13:53:47 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430827, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:47.909343 27871 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:53:47.910281 27871 webserver.cc:460] Webserver started at http://127.10.236.194:42661/ using document root <none> and password file <none>
I20260926 13:53:47.910925 27871 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:47.910989 27871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:47.915053 27871 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:53:47.917336 27887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:47.919356 27871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:53:47.919519 27871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "359713cadb03402a9e96cd6f9416e2ee"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "8d00e1ca0d1f2e8ccff4ae3e97deda5f"
server_key_iv: "615100bdf380ab5969a430b2ee8c9f2a"
server_key_version: "encryptionkey@0"
I20260926 13:53:47.920372 27871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:47.936621 27871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:47.940250 27871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:47.940490 27871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:47.941193 27871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:47.942077 27871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:53:47.942152 27871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:47.942224 27871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:47.942270 27871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:47.953068 27871 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:44475
I20260926 13:53:47.953153 28000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:44475 every 8 connection(s)
I20260926 13:53:47.954159 27871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:53:47.958500 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:47.956493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:54649)
0926 13:53:47.957073 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:47.957201 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":126,"threads_started":1}
W20260926 13:53:47.959031 28001 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:44051 (0 consecutive failures): Network error: Failed to ping master at 127.10.236.254:44051: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
I20260926 13:53:47.959861 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:47.959583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:37019)
0926 13:53:47.959730 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:47.959800 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 13:53:47.960665 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:47.960409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:33011)
0926 13:53:47.960553 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:47.960603 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260926 13:53:47.960901 28001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:53:47.961637 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 27871
I20260926 13:53:47.962317 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:34963
--local_ip_for_outbound_sockets=127.10.236.195
--tserver_master_addrs=127.10.236.254:44051
--webserver_port=42827
--webserver_interface=127.10.236.195
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:53:48.078078 28004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:53:48.078363 28004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:53:48.078449 28004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:53:48.082722 28004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:53:48.082803 28004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:53:48.082865 28004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:53:48.088896 28004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:34963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=42827
--tserver_master_addrs=127.10.236.254:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:53:48.090059 28004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:53:48.091166 28004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:53:48.102116 28009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:48.102210 28004 server_base.cc:1034] running on GCE node
W20260926 13:53:48.102133 28010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:53:48.102133 28012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:53:48.102869 28004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:53:48.103572 28004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:53:48.104761 28004 hybrid_clock.cc:648] HybridClock initialized: now 1790430828104731 us; error 46 us; skew 500 ppm
Sep 26 13:53:48 dist-test-slave-r34j krb5kdc[26553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430828, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:53:48.107870 28004 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:53:48.108906 28004 webserver.cc:460] Webserver started at http://127.10.236.195:42827/ using document root <none> and password file <none>
I20260926 13:53:48.109584 28004 fs_manager.cc:359] Metadata directory not provided
I20260926 13:53:48.109663 28004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:53:48.113904 28004 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:53:48.116659 28019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:48.118183 28004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:53:48.118432 28004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "3782b56c1afa48da8461fd773ff1b746"
format_stamp: "Formatted at 2026-09-26 13:53:05 on dist-test-slave-r34j"
server_key: "7c79b7e85c1afb7068618e7f5765fae1"
server_key_iv: "fbb69a86152f18f50609eafbf0e31fc6"
server_key_version: "encryptionkey@0"
I20260926 13:53:48.119033 28004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:53:48.129628 28004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:53:48.133051 28004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:53:48.133271 28004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:53:48.133878 28004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:53:48.135061 28026 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 13:53:48.139366 28026 ts_tablet_manager.cc:1852] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 13:53:48.139915 28026 ts_tablet_manager.cc:1905] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:53:48.144249 28026 ts_tablet_manager.cc:1918] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:53:48.144372 28026 log.cc:1196] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:48.144726 28026 ts_tablet_manager.cc:1939] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 3782b56c1afa48da8461fd773ff1b746: Deleting consensus metadata
I20260926 13:53:48.145399 28004 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 13:53:48.145499 28004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.000s sys 0.000s
I20260926 13:53:48.145673 28004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:53:48.145737 28004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:53:48.155773 28004 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:34963
I20260926 13:53:48.155836 28133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:34963 every 8 connection(s)
I20260926 13:53:48.156744 28004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:53:48.159935 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28004
I20260926 13:53:48.160821 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.159171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:57605)
0926 13:53:48.159833 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.159996 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":127,"threads_started":1}
W20260926 13:53:48.161453 28134 heartbeater.cc:644] Failed to heartbeat to 127.10.236.254:44051 (0 consecutive failures): Network error: Failed to ping master at 127.10.236.254:44051: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
I20260926 13:53:48.162185 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.161938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:50519)
0926 13:53:48.162065 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.162127 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 13:53:48.162989 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.162767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:58907)
0926 13:53:48.162884 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.162929 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 13:53:48.163245 28134 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:53:48.829433 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.828940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:60471)
0926 13:53:48.829240 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.829311 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":75,"threads_started":1}
I20260926 13:53:48.929299 28139 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:53:48.929523 28139 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:48.930744 28139 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963), 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
Sep 26 13:53:48 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430827, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.195@KRBTEST.COM
Sep 26 13:53:48 dist-test-slave-r34j krb5kdc[26553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430827, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.194@KRBTEST.COM
I20260926 13:53:48.951453 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.931469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:54871 (local address 127.10.236.195:34963)
0926 13:53:48.931817 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:48.931829 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:48.931870 (+ 41us) server_negotiation.cc:407] Connection header received
0926 13:53:48.932799 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:48.932807 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:48.933301 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:48.933645 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:48.934950 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.936827 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.939241 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.940202 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.940423 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:48.944955 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:48.944961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:48.944977 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:48.945015 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:48.948241 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:48.949173 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:48.949179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:48.949184 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:48.949337 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:48.949734 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:48.949741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:48.949743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:48.950221 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:48.950375 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:48.951004 (+ 629us) server_negotiation.cc:299] Negotiation successful
0926 13:53:48.951305 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":84,"threads_started":1}
I20260926 13:53:48.951426 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.931079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:34963 (local address 127.10.236.193:54871)
0926 13:53:48.931275 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.931291 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:48.931418 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:48.933746 (+ 2328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:48.933751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:48.934117 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:48.934579 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:48.934594 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.937096 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.937099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:48.939014 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:48.939030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.940443 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.940448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:48.940894 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:48.941897 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:48.941932 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:48.944603 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:48.948469 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:48.948473 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:48.948484 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:48.949005 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:48.949457 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:48.949462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:48.949468 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:48.949625 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:48.950359 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:48.950366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:48.950591 (+ 225us) client_negotiation.cc:769] Sending connection context
0926 13:53:48.950866 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 13:53:48.951211 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":307,"spinlock_wait_cycles":28800}
I20260926 13:53:48.951665 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.931126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:44475 (local address 127.10.236.193:44651)
0926 13:53:48.931878 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.931894 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:53:48.932023 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:53:48.934912 (+ 2889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:53:48.934915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:53:48.935091 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:53:48.935198 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:48.935204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.937497 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.937502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:48.939018 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:53:48.939030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.940443 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.940448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:53:48.940855 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:48.941897 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:53:48.941932 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:53:48.944987 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:53:48.948462 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:48.948469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:48.948484 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:48.948996 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:48.949472 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:53:48.949480 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:53:48.949482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:53:48.949760 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:53:48.950601 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:53:48.950605 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:53:48.950706 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 13:53:48.951058 (+ 352us) client_negotiation.cc:240] Negotiation successful
0926 13:53:48.951382 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":243,"spinlock_wait_cycles":29696,"thread_start_us":105,"threads_started":1}
I20260926 13:53:48.951954 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.931262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44651 (local address 127.10.236.194:44475)
0926 13:53:48.931780 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0926 13:53:48.931796 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:53:48.931997 (+ 201us) server_negotiation.cc:407] Connection header received
0926 13:53:48.932793 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:53:48.932802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:53:48.933301 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:53:48.933644 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:53:48.935376 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.937260 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.939283 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:53:48.940202 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:53:48.940423 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:53:48.945256 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:53:48.945265 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:53:48.945283 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:53:48.945326 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:53:48.948246 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:48.949170 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:48.949176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:48.949181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:48.949336 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:53:48.949892 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:53:48.949896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:53:48.949897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:53:48.950460 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:53:48.950621 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 13:53:48.951378 (+ 757us) server_negotiation.cc:299] Negotiation successful
0926 13:53:48.951743 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":72,"threads_started":1}
W20260926 13:53:48.954867 27753 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
W20260926 13:53:48.955073 27755 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:48.955159 27753 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b; no voters: 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746
I20260926 13:53:48.959719 28139 raft_consensus.cc:2726] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:53:48.962146 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:48.961581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:36497)
0926 13:53:48.961961 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:48.962033 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":79,"threads_started":1}
I20260926 13:53:49.164516 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:49.164016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:51743)
0926 13:53:49.164335 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:49.164399 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260926 13:53:49.830942 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:49.830419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:51667)
0926 13:53:49.830704 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:49.830841 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":78,"threads_started":1}
I20260926 13:53:49.963822 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:49.963242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:53191)
0926 13:53:49.963646 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:49.963717 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":78,"threads_started":1}
I20260926 13:53:50.165932 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:50.165470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:38835)
0926 13:53:50.165776 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:50.165832 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":86,"threads_started":1}
I20260926 13:53:50.573261 28148 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:53:50.573426 28148 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:50.573937 28148 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963), 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
W20260926 13:53:50.574921 27753 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
W20260926 13:53:50.575003 27755 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:50.575045 27753 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b; no voters: 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746
I20260926 13:53:50.575199 28148 raft_consensus.cc:2726] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:53:50.832544 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:50.832007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:44619)
0926 13:53:50.832350 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:50.832422 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
I20260926 13:53:50.965482 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:50.964763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:55605)
0926 13:53:50.965228 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:50.965351 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":99,"threads_started":1}
I20260926 13:53:51.167448 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:51.166847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:50975)
0926 13:53:51.167251 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:51.167339 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":148,"threads_started":1}
I20260926 13:53:51.834043 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:51.833610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:34555)
0926 13:53:51.833894 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:51.833946 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":121,"threads_started":1}
I20260926 13:53:51.966998 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:51.966490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:45575)
0926 13:53:51.966834 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:51.966894 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
I20260926 13:53:52.168890 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:52.168426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:60237)
0926 13:53:52.168730 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:52.168793 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
I20260926 13:53:52.835572 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:52.834983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:55889)
0926 13:53:52.835297 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:52.835385 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":79,"threads_started":1}
I20260926 13:53:52.968468 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:52.968006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:39645)
0926 13:53:52.968304 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:52.968368 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260926 13:53:53.170578 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:53.169911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:35299)
0926 13:53:53.170225 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:53.170290 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":82,"threads_started":1}
I20260926 13:53:53.784828 28158 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:53:53.785140 28158 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:53.785907 28158 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963), 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
W20260926 13:53:53.787176 27753 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
W20260926 13:53:53.787351 27755 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:53.787526 27755 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b; no voters: 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746
I20260926 13:53:53.787793 28158 raft_consensus.cc:2726] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:53:53.837028 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:53.836546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:50133)
0926 13:53:53.836862 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:53.836926 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":82,"threads_started":1}
I20260926 13:53:53.970077 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:53.969521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:33017)
0926 13:53:53.969837 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:53.969917 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":83,"threads_started":1}
I20260926 13:53:54.172300 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:54.171631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:38915)
0926 13:53:54.172146 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:54.172203 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":103,"threads_started":1}
I20260926 13:53:54.838344 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:54.837907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:49173)
0926 13:53:54.838188 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:54.838248 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260926 13:53:54.971712 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:54.971004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:33567)
0926 13:53:54.971442 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:54.971553 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":89,"threads_started":1}
I20260926 13:53:55.173810 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:55.173308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:53019)
0926 13:53:55.173649 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:55.173712 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":81,"threads_started":1}
I20260926 13:53:55.839691 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:55.839194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:33041)
0926 13:53:55.839521 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:55.839586 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":84,"threads_started":1}
I20260926 13:53:55.973232 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:55.972608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:42709)
0926 13:53:55.973010 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:55.973097 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
I20260926 13:53:56.175438 28167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:56.174843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:56707)
0926 13:53:56.175157 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:56.175310 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":84,"threads_started":1}
I20260926 13:53:56.665915 28168 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:53:56.666074 28168 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:53:56.666543 28168 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963), 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
W20260926 13:53:56.667336 27753 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
W20260926 13:53:56.667449 27755 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:53:56.667580 27755 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b; no voters: 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746
I20260926 13:53:56.667730 28168 raft_consensus.cc:2726] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:53:56.841164 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:56.840663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:44571)
0926 13:53:56.840973 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:56.841036 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":89,"threads_started":1}
I20260926 13:53:56.974700 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:56.974138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:44269)
0926 13:53:56.974444 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:56.974547 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":90,"threads_started":1}
I20260926 13:53:57.176739 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:57.176313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:52465)
0926 13:53:57.176578 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:57.176643 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260926 13:53:57.842898 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:57.842270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:54779)
0926 13:53:57.842687 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:57.842764 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":119,"threads_started":1}
I20260926 13:53:57.976500 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:57.975934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:41917)
0926 13:53:57.976265 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:57.976325 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":110,"threads_started":1}
I20260926 13:53:58.178000 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:58.177562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:39241)
0926 13:53:58.177846 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:58.177906 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260926 13:53:58.844185 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:58.843771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:40171)
0926 13:53:58.844032 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:58.844090 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":113,"threads_started":1}
I20260926 13:53:58.977830 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:58.977385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:57745)
0926 13:53:58.977662 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:58.977732 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
I20260926 13:53:59.179359 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:59.178907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:54621)
0926 13:53:59.179182 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:59.179256 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260926 13:53:59.845436 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:59.845004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:56617)
0926 13:53:59.845282 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:59.845343 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
I20260926 13:53:59.979266 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:53:59.978779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:35009)
0926 13:53:59.979099 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0926 13:53:59.979165 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":77,"threads_started":1}
I20260926 13:54:00.180794 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:00.180289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:56683)
0926 13:54:00.180614 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:00.180681 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":81,"threads_started":1}
I20260926 13:54:00.238024 28181 raft_consensus.cc:482] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:00.238198 28181 raft_consensus.cc:504] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3782b56c1afa48da8461fd773ff1b746" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34963 } } peers { permanent_uuid: "359713cadb03402a9e96cd6f9416e2ee" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 44475 } } peers { permanent_uuid: "2773e3cc6ad143bd8fae5ac104378c8b" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 35817 } }
I20260926 13:54:00.238657 28181 leader_election.cc:288] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963), 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475)
W20260926 13:54:00.239512 27755 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 359713cadb03402a9e96cd6f9416e2ee (127.10.236.194:44475): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
W20260926 13:54:00.239609 27753 leader_election.cc:341] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3782b56c1afa48da8461fd773ff1b746 (127.10.236.195:34963): Not found: Tablet not found: 3e12ec96ea554ee3bfd2b4a33fa9b5a8
I20260926 13:54:00.239719 27753 leader_election.cc:302] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2773e3cc6ad143bd8fae5ac104378c8b; no voters: 359713cadb03402a9e96cd6f9416e2ee, 3782b56c1afa48da8461fd773ff1b746
I20260926 13:54:00.239868 28181 raft_consensus.cc:2726] T 3e12ec96ea554ee3bfd2b4a33fa9b5a8 P 2773e3cc6ad143bd8fae5ac104378c8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:54:00.846684 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:00.846254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:44999)
0926 13:54:00.846509 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:00.846586 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260926 13:54:00.980597 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:00.980167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:54615)
0926 13:54:00.980429 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:00.980490 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
I20260926 13:54:01.182134 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:01.181755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:60765)
0926 13:54:01.181979 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:01.182034 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260926 13:54:01.848065 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:01.847662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:44387)
0926 13:54:01.847911 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:01.847966 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260926 13:54:01.981900 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:01.981496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:57135)
0926 13:54:01.981745 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:01.981801 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260926 13:54:02.183583 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:02.183070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.195:40135)
0926 13:54:02.183359 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:02.183423 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260926 13:54:02.849470 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:02.848988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.193:41639)
0926 13:54:02.849297 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:02.849369 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":75,"threads_started":1}
I20260926 13:54:02.983305 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:02.982846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:44051 (local address 127.10.236.194:56065)
0926 13:54:02.983142 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:02.983205 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.254:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":70,"threads_started":1}
I20260926 13:54:03.164928 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 27697
I20260926 13:54:03.171067 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 27871
I20260926 13:54:03.176662 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28004
2026-09-26T13:54:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59671 ms)
[----------] 30 tests from SecurityITest (248942 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:03 dist-test-slave-r34j krb5kdc[28192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:03 dist-test-slave-r34j krb5kdc[28192](info): set up 2 sockets
Sep 26 13:54:03 dist-test-slave-r34j krb5kdc[28192](info): commencing operation
krb5kdc: starting...
W20260926 13:54:04.231370 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:04 dist-test-slave-r34j krb5kdc[28192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:04.360222 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33517
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:33517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:04.471232 28205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:04.471624 28205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:04.471712 28205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:04.475693 28205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:04.475785 28205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:04.475823 28205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:04.475872 28205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:04.475910 28205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:04.481174 28205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:33517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:04.482209 28205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:04.483204 28205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:04.491580 28213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:04.491791 28210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:04.491667 28205 server_base.cc:1034] running on GCE node
W20260926 13:54:04.491662 28211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:04.492496 28205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:04.493592 28205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:04.494797 28205 hybrid_clock.cc:648] HybridClock initialized: now 1790430844494782 us; error 26 us; skew 500 ppm
Sep 26 13:54:04 dist-test-slave-r34j krb5kdc[28192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430844, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:04.497624 28205 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:04.498517 28205 webserver.cc:460] Webserver started at http://127.10.236.254:37419/ using document root <none> and password file <none>
I20260926 13:54:04.499070 28205 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:04.499155 28205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:04.499392 28205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:04.501623 28205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "641af266617b408a90997a3e9f6b6d49"
format_stamp: "Formatted at 2026-09-26 13:54:04 on dist-test-slave-r34j"
server_key: "a8e6feb2a5740b0f7e4c815c69880e0b"
server_key_iv: "2ae3ce0459b06d800239c3f016544d8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:04.502187 28205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "641af266617b408a90997a3e9f6b6d49"
format_stamp: "Formatted at 2026-09-26 13:54:04 on dist-test-slave-r34j"
server_key: "a8e6feb2a5740b0f7e4c815c69880e0b"
server_key_iv: "2ae3ce0459b06d800239c3f016544d8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:04.505919 28205 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 13:54:04.508437 28220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:04.509873 28205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 13:54:04.510035 28205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "641af266617b408a90997a3e9f6b6d49"
format_stamp: "Formatted at 2026-09-26 13:54:04 on dist-test-slave-r34j"
server_key: "a8e6feb2a5740b0f7e4c815c69880e0b"
server_key_iv: "2ae3ce0459b06d800239c3f016544d8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:04.510155 28205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:04.531348 28205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:04.534665 28205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:04.534931 28205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:04.542341 28205 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:33517
I20260926 13:54:04.542395 28272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:33517 every 8 connection(s)
I20260926 13:54:04.543291 28205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:04.546041 28273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:04.546833 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28205
I20260926 13:54:04.547008 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:04.547307 11187 external_mini_cluster.cc:1428] Setting key 82ccd4988f5e21255466ab7643a22421
I20260926 13:54:04.552600 28273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49: Bootstrap starting.
I20260926 13:54:04.555352 28273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:04.556350 28273 log.cc:824] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:54:04 dist-test-slave-r34j krb5kdc[28192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:04.558382 28273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49: No bootstrap required, opened a new log
I20260926 13:54:04.561993 28273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641af266617b408a90997a3e9f6b6d49" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33517 } }
I20260926 13:54:04.562239 28273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:04.562320 28273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641af266617b408a90997a3e9f6b6d49, State: Initialized, Role: FOLLOWER
I20260926 13:54:04.562356 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:04.548958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.10.236.254:33517)
0926 13:54:04.549494 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:04.549509 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:04.549545 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:54:04.550171 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:04.550188 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:04.550558 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:04.550858 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:04.551753 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:04.552816 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:04.553985 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:04.554471 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:04.554597 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:04.557674 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:04.557679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:04.557692 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:04.557718 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:04.559462 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:04.559950 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:04.559956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:04.559962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:04.560074 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:04.560342 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:04.560347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:04.560349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:04.560747 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:04.561019 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:04.561277 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 13:54:04.561570 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":184,"threads_started":1}
I20260926 13:54:04.562842 28273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641af266617b408a90997a3e9f6b6d49" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33517 } }
I20260926 13:54:04.563066 28273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:04.563134 28273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:04.563225 28273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:04.564208 28273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641af266617b408a90997a3e9f6b6d49" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33517 } }
I20260926 13:54:04.564587 28273 leader_election.cc:302] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 641af266617b408a90997a3e9f6b6d49; no voters:
I20260926 13:54:04.564909 28273 leader_election.cc:288] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:04.565025 28278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:04.565258 28278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [term 1 LEADER]: Becoming Leader. State: Replica: 641af266617b408a90997a3e9f6b6d49, State: Running, Role: LEADER
I20260926 13:54:04.565583 28278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641af266617b408a90997a3e9f6b6d49" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33517 } }
I20260926 13:54:04.566076 28273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:04.567186 28280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 641af266617b408a90997a3e9f6b6d49. Latest consensus state: current_term: 1 leader_uuid: "641af266617b408a90997a3e9f6b6d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641af266617b408a90997a3e9f6b6d49" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33517 } } }
I20260926 13:54:04.567323 28280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:04.567669 28279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "641af266617b408a90997a3e9f6b6d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641af266617b408a90997a3e9f6b6d49" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33517 } } }
I20260926 13:54:04.567782 28287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:04.567809 28279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:04.571028 28287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:04.576177 28287 catalog_manager.cc:1347] Generated new cluster ID: 796887b1559c4ff29c96010c3d9a4266
I20260926 13:54:04.576241 28287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:04.588078 28287 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:04.589551 28287 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:04.595821 28287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49: Generated new TSK 0
I20260926 13:54:04.596701 28287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:54:04.631290 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33517
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:04.744030 28299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:04.744287 28299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:04.744331 28299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:04.748368 28299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:04.748432 28299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:04.748513 28299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:54:04.753770 28299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:04.754813 28299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:04.755920 28299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:04.765573 28305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:04.765602 28307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:04.765573 28304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:04.765980 28299 server_base.cc:1034] running on GCE node
I20260926 13:54:04.766480 28299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:04.767177 28299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:04.768342 28299 hybrid_clock.cc:648] HybridClock initialized: now 1790430844768304 us; error 67 us; skew 500 ppm
Sep 26 13:54:04 dist-test-slave-r34j krb5kdc[28192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430844, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:04.771207 28299 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:54:04.772107 28299 webserver.cc:460] Webserver started at http://127.10.236.193:46661/ using document root <none> and password file <none>
I20260926 13:54:04.772631 28299 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:04.772687 28299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:04.772850 28299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:04.774850 28299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "4008c80caa334d378a18fe7c5f2a4699"
format_stamp: "Formatted at 2026-09-26 13:54:04 on dist-test-slave-r34j"
server_key: "05aae0b8b3ee476d97ebba8eecc66149"
server_key_iv: "c85cff1030c3263eb53edbf8010208a4"
server_key_version: "encryptionkey@0"
I20260926 13:54:04.775357 28299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "4008c80caa334d378a18fe7c5f2a4699"
format_stamp: "Formatted at 2026-09-26 13:54:04 on dist-test-slave-r34j"
server_key: "05aae0b8b3ee476d97ebba8eecc66149"
server_key_iv: "c85cff1030c3263eb53edbf8010208a4"
server_key_version: "encryptionkey@0"
I20260926 13:54:04.779089 28299 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:54:04.781611 28314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:04.782974 28299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:04.783141 28299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "4008c80caa334d378a18fe7c5f2a4699"
format_stamp: "Formatted at 2026-09-26 13:54:04 on dist-test-slave-r34j"
server_key: "05aae0b8b3ee476d97ebba8eecc66149"
server_key_iv: "c85cff1030c3263eb53edbf8010208a4"
server_key_version: "encryptionkey@0"
I20260926 13:54:04.783265 28299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:04.818398 28299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:04.822029 28299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:04.822253 28299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:04.822882 28299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:04.823825 28299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:04.823897 28299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:04.823972 28299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:04.824025 28299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:04.833563 28299 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:39115
I20260926 13:54:04.833590 28427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:39115 every 8 connection(s)
I20260926 13:54:04.834508 28299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:54:04.838371 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28299
I20260926 13:54:04.838514 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:54:04.838829 11187 external_mini_cluster.cc:1428] Setting key 2f80ca9299c46d47bdc190a4c6ec4b63
Sep 26 13:54:04 dist-test-slave-r34j krb5kdc[28192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430844, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
I20260926 13:54:04.850163 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:04.836396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:50213 (local address 127.10.236.254:33517)
0926 13:54:04.836543 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:04.836549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:04.837378 (+ 829us) server_negotiation.cc:407] Connection header received
0926 13:54:04.838757 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:04.838762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:04.838821 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:04.838926 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:04.840504 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:04.841406 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:04.842646 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:04.843105 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:04.843239 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:04.846152 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:04.846155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:04.846157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:04.846182 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:04.847754 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:04.848396 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:04.848405 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:04.848408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:04.848495 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:04.848939 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:04.848941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:04.848943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:04.849132 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:04.849212 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:04.849905 (+ 693us) server_negotiation.cc:299] Negotiation successful
0926 13:54:04.850024 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:54:04.851032 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:04.836809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33517 (local address 127.10.236.193:50213)
0926 13:54:04.837235 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:04.837275 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:04.838527 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:04.839140 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:04.839148 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:04.839673 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:04.840334 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:04.840357 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:04.841565 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:04.841567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:04.842512 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:04.842524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:04.843252 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:04.843254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:04.843392 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:04.844162 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:04.844187 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:04.845937 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:04.847869 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:04.847877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:04.847896 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:04.848282 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:04.848601 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:04.848607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:04.848611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:04.848805 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:04.849241 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:04.849247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:04.849589 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 13:54:04.849823 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 13:54:04.850115 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260926 13:54:04.852408 28428 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33517
I20260926 13:54:04.852669 28428 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:54:04.853245 28428 heartbeater.cc:511] Master 127.10.236.254:33517 requested a full tablet report, sending...
I20260926 13:54:04.854787 28235 ts_manager.cc:194] Registered new tserver with Master: 4008c80caa334d378a18fe7c5f2a4699 (127.10.236.193:39115)
I20260926 13:54:04.856367 28235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:50213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:54:04.870788 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33517
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:04.987982 28433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:04.988298 28433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:04.988371 28433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:04.993078 28433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:04.993191 28433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:04.993299 28433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:54:04.999347 28433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:05.000550 28433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:05.001801 28433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:05.012962 28438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:05.012955 28439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:05.013149 28433 server_base.cc:1034] running on GCE node
W20260926 13:54:05.012979 28441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:05.013759 28433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:05.014530 28433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:05.015730 28433 hybrid_clock.cc:648] HybridClock initialized: now 1790430845015698 us; error 53 us; skew 500 ppm
Sep 26 13:54:05 dist-test-slave-r34j krb5kdc[28192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430845, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:05.018914 28433 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:54:05.019980 28433 webserver.cc:460] Webserver started at http://127.10.236.194:45429/ using document root <none> and password file <none>
I20260926 13:54:05.020567 28433 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:05.020654 28433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:05.020880 28433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:05.022953 28433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "3930b3cc68bb4ae8a65a78a630284fa4"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "9bfb1767c18b3e3e4b161a5b2a4bdf6f"
server_key_iv: "b6b2f7ce2aea79381d6b696bd1bd3da0"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.023579 28433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "3930b3cc68bb4ae8a65a78a630284fa4"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "9bfb1767c18b3e3e4b161a5b2a4bdf6f"
server_key_iv: "b6b2f7ce2aea79381d6b696bd1bd3da0"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.027638 28433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:54:05.030491 28448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.031932 28433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:05.032089 28433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "3930b3cc68bb4ae8a65a78a630284fa4"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "9bfb1767c18b3e3e4b161a5b2a4bdf6f"
server_key_iv: "b6b2f7ce2aea79381d6b696bd1bd3da0"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.032218 28433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:05.056752 28433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:05.060242 28433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:05.060411 28433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:05.060950 28433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:05.061900 28433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:05.061975 28433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.062065 28433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:05.062094 28433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.071830 28433 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:45479
I20260926 13:54:05.071897 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:45479 every 8 connection(s)
I20260926 13:54:05.072844 28433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:54:05.078053 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28433
I20260926 13:54:05.078205 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:54:05.078495 11187 external_mini_cluster.cc:1428] Setting key b1d13d4deba11414613c30710061f545
Sep 26 13:54:05 dist-test-slave-r34j krb5kdc[28192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430845, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:05.087603 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.074802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:54195 (local address 127.10.236.254:33517)
0926 13:54:05.074944 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.074947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.075735 (+ 788us) server_negotiation.cc:407] Connection header received
0926 13:54:05.077120 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.077124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.077187 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.077291 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:05.078533 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.079082 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.080283 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.080504 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.080614 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.083635 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:05.083639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:05.083641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:05.083667 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:05.085275 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:05.085843 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:05.085853 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:05.085855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:05.085937 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:05.086306 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:05.086310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:05.086312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:05.086509 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:05.086632 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.087244 (+ 612us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.087388 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:54:05.088651 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.075099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33517 (local address 127.10.236.194:54195)
0926 13:54:05.075591 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.075635 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.076879 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.077460 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.077467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.077822 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:05.078386 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.078402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.079209 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.079212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.080156 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.080167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.080652 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.080656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.080827 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.081447 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:05.081474 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:05.083390 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:05.085404 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:05.085411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:05.085427 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:05.085727 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:05.086043 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:05.086047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:05.086050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:05.086201 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:05.086629 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:05.086637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:05.086958 (+ 321us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.087188 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.087590 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":119,"threads_started":1}
I20260926 13:54:05.089944 28562 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33517
I20260926 13:54:05.090359 28562 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:05.091022 28562 heartbeater.cc:511] Master 127.10.236.254:33517 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:54:05.092447 28235 ts_manager.cc:194] Registered new tserver with Master: 3930b3cc68bb4ae8a65a78a630284fa4 (127.10.236.194:45479)
I20260926 13:54:05.093307 28235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:54195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:54:05.111732 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33517
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:05.224442 28567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:05.224709 28567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:05.224751 28567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:05.228760 28567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:05.228816 28567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:05.228873 28567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:54:05.234002 28567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:05.234948 28567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:05.236007 28567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:05.245553 28575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:05.245472 28572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:05.245734 28573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:05.245872 28567 server_base.cc:1034] running on GCE node
I20260926 13:54:05.246279 28567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:05.246996 28567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:05.248209 28567 hybrid_clock.cc:648] HybridClock initialized: now 1790430845248175 us; error 47 us; skew 500 ppm
Sep 26 13:54:05 dist-test-slave-r34j krb5kdc[28192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430845, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:05.251266 28567 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:54:05.252213 28567 webserver.cc:460] Webserver started at http://127.10.236.195:44901/ using document root <none> and password file <none>
I20260926 13:54:05.252781 28567 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:05.252869 28567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:05.253093 28567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:05.255141 28567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "fc29872ef0b74f11b95ef94dc69fded6"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "8d826822777cf96bb77c43cc24d2c9ae"
server_key_iv: "09e20ee9fc7b1dc650675bf651e5ccbe"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.255771 28567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "fc29872ef0b74f11b95ef94dc69fded6"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "8d826822777cf96bb77c43cc24d2c9ae"
server_key_iv: "09e20ee9fc7b1dc650675bf651e5ccbe"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.259608 28567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 13:54:05.262133 28582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.263442 28567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:05.263643 28567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "fc29872ef0b74f11b95ef94dc69fded6"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "8d826822777cf96bb77c43cc24d2c9ae"
server_key_iv: "09e20ee9fc7b1dc650675bf651e5ccbe"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.263756 28567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:05.278374 28567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:05.281658 28567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:05.281857 28567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:05.282435 28567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:05.283264 28567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:05.283334 28567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.283401 28567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:05.283450 28567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.292464 28567 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:40147
I20260926 13:54:05.292505 28695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:40147 every 8 connection(s)
I20260926 13:54:05.293363 28567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:54:05.298596 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28567
I20260926 13:54:05.298763 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:54:05.299089 11187 external_mini_cluster.cc:1428] Setting key a7a842085d56d3419d5669e60ef8e384
Sep 26 13:54:05 dist-test-slave-r34j krb5kdc[28192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430845, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:05.308027 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.295198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:57493 (local address 127.10.236.254:33517)
0926 13:54:05.295369 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.295374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.296185 (+ 811us) server_negotiation.cc:407] Connection header received
0926 13:54:05.297639 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.297642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.297693 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.297779 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:05.299183 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.300063 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.301475 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.301716 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.301804 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.304423 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:05.304427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:05.304430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:05.304457 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:05.306007 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:05.306492 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:05.306495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:05.306496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:05.306588 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:05.306889 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:05.306892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:05.306893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:05.307106 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:05.307189 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.307776 (+ 587us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.307878 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 13:54:05.308867 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.295578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33517 (local address 127.10.236.195:57493)
0926 13:54:05.296032 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.296077 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.297363 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.297959 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.297969 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.298361 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:05.299022 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.299039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.300202 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.300204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.301361 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.301371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.301845 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.301848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.301995 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.302606 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:05.302624 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:05.304257 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:05.306121 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:05.306125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:05.306138 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:05.306399 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:05.306671 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:05.306673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:05.306675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:05.306808 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:05.307200 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:05.307205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:05.307442 (+ 237us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.307717 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.307985 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":110,"threads_started":1}
I20260926 13:54:05.309973 28696 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33517
I20260926 13:54:05.310187 28696 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:05.310612 28696 heartbeater.cc:511] Master 127.10.236.254:33517 requested a full tablet report, sending...
I20260926 13:54:05.311583 28235 ts_manager.cc:194] Registered new tserver with Master: fc29872ef0b74f11b95ef94dc69fded6 (127.10.236.195:40147)
I20260926 13:54:05.312306 28235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:57493
I20260926 13:54:05.313103 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:54:05.322736 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.315206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.10.236.254:33517)
0926 13:54:05.315362 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.315367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.315521 (+ 154us) server_negotiation.cc:407] Connection header received
0926 13:54:05.315654 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.315656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.315693 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.315754 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:05.316431 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.316942 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.317776 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.318009 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.318091 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.319575 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:05.319578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:05.319581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:05.319610 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:05.320950 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:05.321368 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:05.321370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:05.321371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:05.321442 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:05.321739 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:05.321742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:05.321743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:05.322024 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:05.322164 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.322375 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.322528 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 13:54:05.325613 28235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:54:05.328766 28235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:54:05.344023 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.338001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.10.236.195:40147)
0926 13:54:05.338371 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.338385 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.338409 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:54:05.338507 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.338512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.338754 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.339338 (+ 584us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:05.339692 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.340947 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.342420 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.343003 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.343151 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.343293 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.343725 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.343841 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":145,"threads_started":1}
I20260926 13:54:05.343936 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.337339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:40147 (local address 127.0.0.1:47862)
0926 13:54:05.337671 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.337719 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.338045 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.339307 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.339313 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.339345 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:05.339552 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.339564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.341115 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.341119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.342290 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.342300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.343362 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.343366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.343476 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.343541 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.343651 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.343772 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":177,"thread_start_us":77,"threads_started":1}
I20260926 13:54:05.344933 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.338774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:39115 (local address 127.0.0.1:45542)
0926 13:54:05.339069 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.339081 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.339156 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.339840 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.339846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.339864 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:05.339941 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.339945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.341172 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.341176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.342956 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.342964 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.344530 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.344533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.344630 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.344660 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.344733 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.344810 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":104,"thread_start_us":88,"threads_started":1}
I20260926 13:54:05.344967 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.337353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:45479 (local address 127.0.0.1:33804)
0926 13:54:05.338767 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.338781 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.338877 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.339546 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.339549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.339570 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:05.339699 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.339707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.341115 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.341119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.342952 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.342964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.344418 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.344422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.344543 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.344570 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.344737 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.344818 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"mutex_wait_us":271,"thread_start_us":127,"threads_started":1}
I20260926 13:54:05.345440 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.338950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.10.236.193:39115)
0926 13:54:05.339215 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.339235 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.339259 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:54:05.339337 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.339341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.339554 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.339709 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:05.340054 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.341017 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.343106 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.343978 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.344122 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.344467 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.345226 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.345327 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260926 13:54:05.345461 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.337438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.10.236.194:45479)
0926 13:54:05.337871 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.337881 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.338979 (+ 1098us) server_negotiation.cc:407] Connection header received
0926 13:54:05.339071 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.339080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.339271 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.339408 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:05.339821 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.340948 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.343110 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.343901 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.344028 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.345152 (+ 1124us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.345209 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.345310 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":154,"threads_started":1}
I20260926 13:54:05.347718 28362 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff63cebe4574e27b567eff117133c0f (DEFAULT_TABLE table=test-table [id=0a98ab50f138407d911590ff66c35001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:05.347724 28630 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff63cebe4574e27b567eff117133c0f (DEFAULT_TABLE table=test-table [id=0a98ab50f138407d911590ff66c35001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:05.348054 28496 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff63cebe4574e27b567eff117133c0f (DEFAULT_TABLE table=test-table [id=0a98ab50f138407d911590ff66c35001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:05.348809 28362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff63cebe4574e27b567eff117133c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:05.348827 28630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff63cebe4574e27b567eff117133c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:05.348904 28496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff63cebe4574e27b567eff117133c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:05.355362 28713 tablet_bootstrap.cc:492] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: Bootstrap starting.
I20260926 13:54:05.356935 28714 tablet_bootstrap.cc:492] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Bootstrap starting.
I20260926 13:54:05.357276 28715 tablet_bootstrap.cc:492] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Bootstrap starting.
I20260926 13:54:05.358096 28713 tablet_bootstrap.cc:654] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:05.359289 28713 log.cc:824] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:05.359398 28715 tablet_bootstrap.cc:654] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:05.359398 28714 tablet_bootstrap.cc:654] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:05.360908 28714 log.cc:824] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:05.360908 28715 log.cc:824] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:05.361289 28713 tablet_bootstrap.cc:492] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: No bootstrap required, opened a new log
I20260926 13:54:05.361498 28713 ts_tablet_manager.cc:1397] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:54:05.363013 28715 tablet_bootstrap.cc:492] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: No bootstrap required, opened a new log
I20260926 13:54:05.363287 28715 ts_tablet_manager.cc:1397] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260926 13:54:05.363389 28714 tablet_bootstrap.cc:492] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: No bootstrap required, opened a new log
I20260926 13:54:05.363631 28714 ts_tablet_manager.cc:1397] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:54:05.365406 28713 raft_consensus.cc:348] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.365913 28713 raft_consensus.cc:374] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:05.366048 28713 raft_consensus.cc:729] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4008c80caa334d378a18fe7c5f2a4699, State: Initialized, Role: FOLLOWER
I20260926 13:54:05.366616 28713 consensus_queue.cc:260] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.367455 28428 heartbeater.cc:503] Master 127.10.236.254:33517 was elected leader, sending a full tablet report...
I20260926 13:54:05.367928 28713 ts_tablet_manager.cc:1428] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:54:05.367884 28714 raft_consensus.cc:348] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.367911 28715 raft_consensus.cc:348] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.368232 28715 raft_consensus.cc:374] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:05.368245 28714 raft_consensus.cc:374] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:05.368309 28715 raft_consensus.cc:729] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc29872ef0b74f11b95ef94dc69fded6, State: Initialized, Role: FOLLOWER
I20260926 13:54:05.368316 28714 raft_consensus.cc:729] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3930b3cc68bb4ae8a65a78a630284fa4, State: Initialized, Role: FOLLOWER
I20260926 13:54:05.368783 28715 consensus_queue.cc:260] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.368795 28714 consensus_queue.cc:260] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.369450 28562 heartbeater.cc:503] Master 127.10.236.254:33517 was elected leader, sending a full tablet report...
I20260926 13:54:05.369786 28714 ts_tablet_manager.cc:1428] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:05.369786 28715 ts_tablet_manager.cc:1428] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:05.370118 28696 heartbeater.cc:503] Master 127.10.236.254:33517 was elected leader, sending a full tablet report...
I20260926 13:54:05.479424 28721 raft_consensus.cc:482] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:05.479668 28721 raft_consensus.cc:504] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.480842 28721 leader_election.cc:288] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3930b3cc68bb4ae8a65a78a630284fa4 (127.10.236.194:45479), 4008c80caa334d378a18fe7c5f2a4699 (127.10.236.193:39115)
I20260926 13:54:05.484968 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.481289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:52073 (local address 127.10.236.194:45479)
0926 13:54:05.481444 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.481449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.481469 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:54:05.481556 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.481562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.481637 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.481719 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:05.482210 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.482827 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.483971 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.484388 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.484483 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.484561 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.484767 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.484837 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:54:05.484968 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.481657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:36779 (local address 127.10.236.193:39115)
0926 13:54:05.481873 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.481878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.481895 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:54:05.481945 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.481947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.481996 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.482081 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:05.482403 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.482911 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.484034 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.484426 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.484575 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.484628 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.484796 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.484859 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 13:54:05.484969 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.481139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:45479 (local address 127.10.236.195:52073)
0926 13:54:05.481315 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.481333 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.481449 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.481899 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.481902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.481924 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:05.482080 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.482089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.482945 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.482952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.483850 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.483860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.484508 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.484514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.484624 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.484659 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.484727 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.484820 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":7}
I20260926 13:54:05.484984 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.481408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:39115 (local address 127.10.236.195:36779)
0926 13:54:05.481776 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.481787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:05.481860 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:05.482115 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:05.482117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:05.482130 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:05.482288 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.482294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.483017 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.483019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.483885 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:05.483894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.484569 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.484573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:05.484662 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.484678 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 13:54:05.484769 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 13:54:05.484837 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":21,"thread_start_us":179,"threads_started":1}
I20260926 13:54:05.485643 28382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff63cebe4574e27b567eff117133c0f" candidate_uuid: "fc29872ef0b74f11b95ef94dc69fded6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4008c80caa334d378a18fe7c5f2a4699" is_pre_election: true
I20260926 13:54:05.485646 28516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff63cebe4574e27b567eff117133c0f" candidate_uuid: "fc29872ef0b74f11b95ef94dc69fded6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" is_pre_election: true
I20260926 13:54:05.485925 28382 raft_consensus.cc:2454] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc29872ef0b74f11b95ef94dc69fded6 in term 0.
I20260926 13:54:05.485935 28516 raft_consensus.cc:2454] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc29872ef0b74f11b95ef94dc69fded6 in term 0.
I20260926 13:54:05.486366 28586 leader_election.cc:302] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4008c80caa334d378a18fe7c5f2a4699, fc29872ef0b74f11b95ef94dc69fded6; no voters:
I20260926 13:54:05.486653 28721 raft_consensus.cc:2781] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:54:05.486728 28721 raft_consensus.cc:482] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:54:05.486781 28721 raft_consensus.cc:3037] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:05.488166 28721 raft_consensus.cc:504] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.488502 28721 leader_election.cc:288] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [CANDIDATE]: Term 1 election: Requested vote from peers 3930b3cc68bb4ae8a65a78a630284fa4 (127.10.236.194:45479), 4008c80caa334d378a18fe7c5f2a4699 (127.10.236.193:39115)
I20260926 13:54:05.488885 28382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff63cebe4574e27b567eff117133c0f" candidate_uuid: "fc29872ef0b74f11b95ef94dc69fded6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4008c80caa334d378a18fe7c5f2a4699"
I20260926 13:54:05.488929 28516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff63cebe4574e27b567eff117133c0f" candidate_uuid: "fc29872ef0b74f11b95ef94dc69fded6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3930b3cc68bb4ae8a65a78a630284fa4"
I20260926 13:54:05.489068 28382 raft_consensus.cc:3037] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:05.489110 28516 raft_consensus.cc:3037] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:05.490103 28382 raft_consensus.cc:2454] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc29872ef0b74f11b95ef94dc69fded6 in term 1.
I20260926 13:54:05.490108 28516 raft_consensus.cc:2454] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc29872ef0b74f11b95ef94dc69fded6 in term 1.
I20260926 13:54:05.490399 28586 leader_election.cc:302] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4008c80caa334d378a18fe7c5f2a4699, fc29872ef0b74f11b95ef94dc69fded6; no voters:
I20260926 13:54:05.490561 28721 raft_consensus.cc:2781] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:05.490823 28721 raft_consensus.cc:686] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 1 LEADER]: Becoming Leader. State: Replica: fc29872ef0b74f11b95ef94dc69fded6, State: Running, Role: LEADER
I20260926 13:54:05.491184 28721 consensus_queue.cc:237] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } }
I20260926 13:54:05.494436 28234 catalog_manager.cc:5495] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc29872ef0b74f11b95ef94dc69fded6 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "fc29872ef0b74f11b95ef94dc69fded6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc29872ef0b74f11b95ef94dc69fded6" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 40147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 } health_report { overall_health: UNKNOWN } } }
I20260926 13:54:05.533360 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.529615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.10.236.195:40147)
0926 13:54:05.529782 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.529788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.529805 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:54:05.530022 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.530025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.530084 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.530218 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:05.530477 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.530978 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.531897 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.532118 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.532219 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.532524 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:05.532966 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:05.533094 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.533141 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.533195 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 13:54:05.539573 28516 raft_consensus.cc:1261] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 1 FOLLOWER]: Refusing update from remote peer fc29872ef0b74f11b95ef94dc69fded6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:05.539572 28382 raft_consensus.cc:1261] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 1 FOLLOWER]: Refusing update from remote peer fc29872ef0b74f11b95ef94dc69fded6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:05.540375 28721 consensus_queue.cc:1035] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3930b3cc68bb4ae8a65a78a630284fa4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 45479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:05.540620 28723 consensus_queue.cc:1035] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4008c80caa334d378a18fe7c5f2a4699" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 13:54:05.544255 28697 tablet.cc:2369] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:54:05.548990 28726 mvcc.cc:205] Tried to move back new op lower bound from 7333604743321878528 to 7333604743134679040. Current Snapshot: MvccSnapshot[applied={T|T < 7333604743321878528}]
I20260926 13:54:05.554353 28728 mvcc.cc:205] Tried to move back new op lower bound from 7333604743321878528 to 7333604743134679040. Current Snapshot: MvccSnapshot[applied={T|T < 7333604743321878528}]
I20260926 13:54:05.558799 28235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:05.558980 28235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:05.561378 28235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 641af266617b408a90997a3e9f6b6d49: Sending DeleteTablet for 3 replicas of tablet 9ff63cebe4574e27b567eff117133c0f
I20260926 13:54:05.562148 28630 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff63cebe4574e27b567eff117133c0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:05 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:47862
I20260926 13:54:05.562198 28362 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff63cebe4574e27b567eff117133c0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:05 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:45542
I20260926 13:54:05.562680 28737 tablet_replica.cc:330] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699: stopping tablet replica
I20260926 13:54:05.562870 28496 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff63cebe4574e27b567eff117133c0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:05 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:33804
I20260926 13:54:05.562983 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28299
I20260926 13:54:05.562987 28737 raft_consensus.cc:2229] T 9ff63cebe4574e27b567eff117133c0f P 4008c80caa334d378a18fe7c5f2a4699 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:05.564018 28738 tablet_replica.cc:330] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: stopping tablet replica
I20260926 13:54:05.564333 28738 raft_consensus.cc:2229] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:05.564664 28738 raft_consensus.cc:2258] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:05.566220 28738 ts_tablet_manager.cc:1905] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:05.569787 28738 ts_tablet_manager.cc:1918] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:05.569895 28738 log.cc:1196] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/9ff63cebe4574e27b567eff117133c0f
I20260926 13:54:05.570261 28738 ts_tablet_manager.cc:1939] T 9ff63cebe4574e27b567eff117133c0f P 3930b3cc68bb4ae8a65a78a630284fa4: Deleting consensus metadata
I20260926 13:54:05.571619 28224 catalog_manager.cc:4841] TS 3930b3cc68bb4ae8a65a78a630284fa4 (127.10.236.194:45479): tablet 9ff63cebe4574e27b567eff117133c0f (table test-table [id=0a98ab50f138407d911590ff66c35001]) successfully deleted
W20260926 13:54:05.572580 28224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:39115 (error 108)
I20260926 13:54:05.572747 28736 tablet_replica.cc:330] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: stopping tablet replica
I20260926 13:54:05.573092 28736 raft_consensus.cc:2229] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:54:05.573486 28736 raft_consensus.cc:2258] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:05.573877 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28433
I20260926 13:54:05.573899 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.573548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:39115 (local address 127.0.0.1:45556)
0926 13:54:05.573734 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:05.573804 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:39115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260926 13:54:05.574923 28224 catalog_manager.cc:4584] TS 4008c80caa334d378a18fe7c5f2a4699 (127.10.236.193:39115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ff63cebe4574e27b567eff117133c0f: Network error: Client connection negotiation failed: client connection to 127.10.236.193:39115: connect: Connection refused (error 111)
I20260926 13:54:05.576483 28736 ts_tablet_manager.cc:1905] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:05.579402 28736 ts_tablet_manager.cc:1918] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:05.579557 28736 log.cc:1196] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/9ff63cebe4574e27b567eff117133c0f
I20260926 13:54:05.579980 28736 ts_tablet_manager.cc:1939] T 9ff63cebe4574e27b567eff117133c0f P fc29872ef0b74f11b95ef94dc69fded6: Deleting consensus metadata
I20260926 13:54:05.580299 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28567
I20260926 13:54:05.585593 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28205
2026-09-26T13:54:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2410 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T13:54:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:05Z Disabled control of system clock
I20260926 13:54:05.618217 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43623
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:54:05.733803 28743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:05.734200 28743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:05.734270 28743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:05.738412 28743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:05.738485 28743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:05.738502 28743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:05.738518 28743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:05.738533 28743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:05.743530 28743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:43623
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:05.744645 28743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:05.746014 28743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:05.754532 28751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:05.754523 28748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:05.754523 28749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:05.754801 28743 server_base.cc:1034] running on GCE node
I20260926 13:54:05.755625 28743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:05.756712 28743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:05.757889 28743 hybrid_clock.cc:648] HybridClock initialized: now 1790430845757855 us; error 42 us; skew 500 ppm
I20260926 13:54:05.760026 28743 webserver.cc:460] Webserver started at http://127.10.236.254:38683/ using document root <none> and password file <none>
I20260926 13:54:05.760620 28743 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:05.760684 28743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:05.760852 28743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:05.762831 28743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "c57e0ec874154a3aad8482c80723e869"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "4163c150ea09c6b6c46b865d0bfa9010"
server_key_iv: "b81f7a0fb7d8c43479e432c1e53a6cfc"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.763388 28743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "c57e0ec874154a3aad8482c80723e869"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "4163c150ea09c6b6c46b865d0bfa9010"
server_key_iv: "b81f7a0fb7d8c43479e432c1e53a6cfc"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.767228 28743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:54:05.769995 28757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:05.771247 28743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 13:54:05.771441 28743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "c57e0ec874154a3aad8482c80723e869"
format_stamp: "Formatted at 2026-09-26 13:54:05 on dist-test-slave-r34j"
server_key: "4163c150ea09c6b6c46b865d0bfa9010"
server_key_iv: "b81f7a0fb7d8c43479e432c1e53a6cfc"
server_key_version: "encryptionkey@0"
I20260926 13:54:05.771608 28743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:05.793994 28743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:05.794761 28743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:05.794950 28743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:05.802387 28809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:43623 every 8 connection(s)
I20260926 13:54:05.802397 28743 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:43623
I20260926 13:54:05.803617 28743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:05.804702 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28743
I20260926 13:54:05.804826 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:05.805136 11187 external_mini_cluster.cc:1428] Setting key 6b49eb7ac023ec9cee41ac7721d0ba3a
I20260926 13:54:05.807422 28810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:05.813020 28810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869: Bootstrap starting.
I20260926 13:54:05.814651 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:05.806680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.10.236.254:43623)
0926 13:54:05.807357 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:05.807375 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:05.807412 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:54:05.808218 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:05.808259 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:05.808688 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:05.809041 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:05.809468 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.810444 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.811314 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:05.811830 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:05.812006 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:05.812911 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:05.812916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:05.812929 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:05.813364 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:05.813435 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:05.813457 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:05.813574 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:05.813687 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 13:54:05.813950 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":154,"threads_started":1}
I20260926 13:54:05.815344 28810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:05.816214 28810 log.cc:824] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:05.818434 28810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869: No bootstrap required, opened a new log
I20260926 13:54:05.822527 28810 raft_consensus.cc:348] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57e0ec874154a3aad8482c80723e869" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43623 } }
I20260926 13:54:05.822844 28810 raft_consensus.cc:374] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:05.822935 28810 raft_consensus.cc:729] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c57e0ec874154a3aad8482c80723e869, State: Initialized, Role: FOLLOWER
I20260926 13:54:05.823473 28810 consensus_queue.cc:260] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57e0ec874154a3aad8482c80723e869" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43623 } }
I20260926 13:54:05.823691 28810 raft_consensus.cc:388] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:05.823763 28810 raft_consensus.cc:482] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:05.823871 28810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:05.824869 28810 raft_consensus.cc:504] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57e0ec874154a3aad8482c80723e869" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43623 } }
I20260926 13:54:05.825237 28810 leader_election.cc:302] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c57e0ec874154a3aad8482c80723e869; no voters:
I20260926 13:54:05.825568 28810 leader_election.cc:288] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:05.825722 28815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:05.826015 28815 raft_consensus.cc:686] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [term 1 LEADER]: Becoming Leader. State: Replica: c57e0ec874154a3aad8482c80723e869, State: Running, Role: LEADER
I20260926 13:54:05.826340 28815 consensus_queue.cc:237] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57e0ec874154a3aad8482c80723e869" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43623 } }
I20260926 13:54:05.826901 28810 sys_catalog.cc:564] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:05.828090 28817 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c57e0ec874154a3aad8482c80723e869. Latest consensus state: current_term: 1 leader_uuid: "c57e0ec874154a3aad8482c80723e869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57e0ec874154a3aad8482c80723e869" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43623 } } }
I20260926 13:54:05.828235 28817 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:05.828429 28816 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c57e0ec874154a3aad8482c80723e869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c57e0ec874154a3aad8482c80723e869" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43623 } } }
I20260926 13:54:05.828608 28816 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:05.828649 28824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:05.832193 28824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:05.838301 28824 catalog_manager.cc:1347] Generated new cluster ID: ecd67920cde746778b9591e15a579222
I20260926 13:54:05.838389 28824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:05.874116 28824 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:05.875161 28824 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:05.879199 28824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869: Generated new TSK 0
I20260926 13:54:05.879931 28824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:05.884896 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43623
--builtin_ntp_servers=127.10.236.212:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:54:06.005671 28834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:06.005959 28834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:06.006001 28834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:06.010169 28834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:06.010233 28834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:06.010291 28834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:54:06.015177 28834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:06.016374 28834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:06.017783 28834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:06.027827 28840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:06.027876 28839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:06.027876 28842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:06.028407 28834 server_base.cc:1034] running on GCE node
I20260926 13:54:06.028882 28834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:06.029621 28834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:06.030794 28834 hybrid_clock.cc:648] HybridClock initialized: now 1790430846030777 us; error 30 us; skew 500 ppm
I20260926 13:54:06.032866 28834 webserver.cc:460] Webserver started at http://127.10.236.193:36685/ using document root <none> and password file <none>
I20260926 13:54:06.033417 28834 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:06.033483 28834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:06.033645 28834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:06.035782 28834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "d897bbc4b76b41eb80ef854afdf84544"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "323ea9e7b7b4f28363dc72bb23550d10"
server_key_iv: "095174199bf0e13cbe5973049fba247a"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.036301 28834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "d897bbc4b76b41eb80ef854afdf84544"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "323ea9e7b7b4f28363dc72bb23550d10"
server_key_iv: "095174199bf0e13cbe5973049fba247a"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.040053 28834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:54:06.042663 28848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.044005 28834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:06.044142 28834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "d897bbc4b76b41eb80ef854afdf84544"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "323ea9e7b7b4f28363dc72bb23550d10"
server_key_iv: "095174199bf0e13cbe5973049fba247a"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.044258 28834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:06.064144 28834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:06.064863 28834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:06.065073 28834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:06.065665 28834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:06.066612 28834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:06.066687 28834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.066771 28834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:06.066813 28834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.075990 28834 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:36947
I20260926 13:54:06.076057 28961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:36947 every 8 connection(s)
I20260926 13:54:06.076936 28834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:54:06.082160 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28834
I20260926 13:54:06.082325 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:54:06.082614 11187 external_mini_cluster.cc:1428] Setting key 181483cd9d9ed8a949f65891097f273a
I20260926 13:54:06.084575 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43623
--builtin_ntp_servers=127.10.236.212:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:54:06.087452 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.078978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:44679 (local address 127.10.236.254:43623)
0926 13:54:06.079145 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.079150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.080215 (+ 1065us) server_negotiation.cc:407] Connection header received
0926 13:54:06.081648 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.081652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.081720 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.081827 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:06.082964 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.083677 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.085077 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.085315 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.085443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.086878 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:06.086881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:06.086883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:06.086896 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:06.086918 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:06.086927 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:06.086994 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.087178 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.087273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:54:06.088434 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.079265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43623 (local address 127.10.236.193:44679)
0926 13:54:06.080038 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.080083 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.081408 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.082052 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.082060 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.082152 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:06.082812 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.082828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.083839 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.083843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.084938 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.084948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.085483 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.085487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.085725 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.086514 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:06.086518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:06.086581 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:06.086698 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:06.087020 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:06.087028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:06.087032 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.087173 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.087468 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":109,"threads_started":1}
I20260926 13:54:06.089809 28962 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43623
I20260926 13:54:06.090099 28962 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:06.090796 28962 heartbeater.cc:511] Master 127.10.236.254:43623 requested a full tablet report, sending...
I20260926 13:54:06.092612 28774 ts_manager.cc:194] Registered new tserver with Master: d897bbc4b76b41eb80ef854afdf84544 (127.10.236.193:36947)
I20260926 13:54:06.093768 28774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.193:44679
W20260926 13:54:06.200240 28965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:06.200546 28965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:06.200616 28965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:06.204666 28965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:06.204744 28965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:06.204830 28965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:54:06.209537 28965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:06.210611 28965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:06.212107 28965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:06.221844 28971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:06.221978 28965 server_base.cc:1034] running on GCE node
W20260926 13:54:06.221841 28973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:06.221841 28970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:06.222596 28965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:06.223275 28965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:06.224449 28965 hybrid_clock.cc:648] HybridClock initialized: now 1790430846224431 us; error 35 us; skew 500 ppm
I20260926 13:54:06.226496 28965 webserver.cc:460] Webserver started at http://127.10.236.194:38193/ using document root <none> and password file <none>
I20260926 13:54:06.227208 28965 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:06.227277 28965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:06.227568 28965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:06.229858 28965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "393cf783b5e345de9e1deb6e6176bd0e"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "72c4203231271dba2097b52f0ce6569b"
server_key_iv: "e63911733adc4592d2848f37fe325f24"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.230470 28965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "393cf783b5e345de9e1deb6e6176bd0e"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "72c4203231271dba2097b52f0ce6569b"
server_key_iv: "e63911733adc4592d2848f37fe325f24"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.234484 28965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260926 13:54:06.237125 28979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.238402 28965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:54:06.238550 28965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "393cf783b5e345de9e1deb6e6176bd0e"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "72c4203231271dba2097b52f0ce6569b"
server_key_iv: "e63911733adc4592d2848f37fe325f24"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.238673 28965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:06.283689 28965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:06.284466 28965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:06.284687 28965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:06.285341 28965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:06.286377 28965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:06.286451 28965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.286525 28965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:06.286577 28965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.296525 28965 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:41479
I20260926 13:54:06.296555 29092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:41479 every 8 connection(s)
I20260926 13:54:06.297609 28965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:54:06.302326 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 28965
I20260926 13:54:06.302479 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:54:06.302845 11187 external_mini_cluster.cc:1428] Setting key 58ee0a181b0d37900abd9f0526cc7cb1
I20260926 13:54:06.305039 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43623
--builtin_ntp_servers=127.10.236.212:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:54:06.308197 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.299587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:42711 (local address 127.10.236.254:43623)
0926 13:54:06.299719 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.299723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.300943 (+ 1220us) server_negotiation.cc:407] Connection header received
0926 13:54:06.302609 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.302613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.302682 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.302765 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:06.303975 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.304647 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.306037 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.306284 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.306403 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.307635 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:06.307639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:06.307642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:06.307662 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:06.307683 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:06.307692 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:06.307773 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.307962 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.308039 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:54:06.309213 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.300071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43623 (local address 127.10.236.194:42711)
0926 13:54:06.300773 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.300816 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.302316 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.302986 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.302996 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.303123 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:06.303818 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.303838 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.304782 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.304786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.305897 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.305907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.306448 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.306452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.306644 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.307319 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:06.307321 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:06.307365 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:06.307436 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:06.307790 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:06.307802 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:06.307813 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.307940 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.308265 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":110,"threads_started":1}
I20260926 13:54:06.310477 29093 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43623
I20260926 13:54:06.310760 29093 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:06.311548 29093 heartbeater.cc:511] Master 127.10.236.254:43623 requested a full tablet report, sending...
I20260926 13:54:06.312813 28774 ts_manager.cc:194] Registered new tserver with Master: 393cf783b5e345de9e1deb6e6176bd0e (127.10.236.194:41479)
I20260926 13:54:06.313735 28774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.194:42711
W20260926 13:54:06.420572 29096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:06.420840 29096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:06.420885 29096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:06.424974 29096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:06.425038 29096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:06.425098 29096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:54:06.430124 29096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:06.431200 29096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:06.432683 29096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:06.442938 29102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:06.442973 29101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:06.442847 29104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:06.443176 29096 server_base.cc:1034] running on GCE node
I20260926 13:54:06.443658 29096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:06.444321 29096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:06.445520 29096 hybrid_clock.cc:648] HybridClock initialized: now 1790430846445489 us; error 38 us; skew 500 ppm
I20260926 13:54:06.447328 29096 webserver.cc:460] Webserver started at http://127.10.236.195:37241/ using document root <none> and password file <none>
I20260926 13:54:06.447989 29096 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:06.448081 29096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:06.448305 29096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:06.450345 29096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "3b654851f0eb4638b5049202af3e2190"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "96fd9f54796277a48fd6e13bb8918024"
server_key_iv: "770433c9ecb5f94acf55bedc81ff84c9"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.450918 29096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "3b654851f0eb4638b5049202af3e2190"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "96fd9f54796277a48fd6e13bb8918024"
server_key_iv: "770433c9ecb5f94acf55bedc81ff84c9"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.454784 29096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:06.457547 29110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.458974 29096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 13:54:06.459123 29096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "3b654851f0eb4638b5049202af3e2190"
format_stamp: "Formatted at 2026-09-26 13:54:06 on dist-test-slave-r34j"
server_key: "96fd9f54796277a48fd6e13bb8918024"
server_key_iv: "770433c9ecb5f94acf55bedc81ff84c9"
server_key_version: "encryptionkey@0"
I20260926 13:54:06.459240 29096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:06.478250 29096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:06.478988 29096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:06.479203 29096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:06.479889 29096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:06.480842 29096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:06.480913 29096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.480991 29096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:06.481031 29096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:06.489729 29096 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:42817
I20260926 13:54:06.489753 29223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:42817 every 8 connection(s)
I20260926 13:54:06.490648 29096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:54:06.491731 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29096
I20260926 13:54:06.491839 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:54:06.492151 11187 external_mini_cluster.cc:1428] Setting key bcd7b57e53485d8ea5fccb1192bbaa0e
I20260926 13:54:06.500635 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.492716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:38601 (local address 127.10.236.254:43623)
0926 13:54:06.492875 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.492879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.494071 (+ 1192us) server_negotiation.cc:407] Connection header received
0926 13:54:06.495578 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.495581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.495633 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.495705 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:06.496592 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.497134 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.498340 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.498564 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.498668 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.500017 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:06.500020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:06.500021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:06.500035 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:06.500059 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:06.500070 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:06.500141 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.500422 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.500504 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:54:06.501600 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.493225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:43623 (local address 127.10.236.195:38601)
0926 13:54:06.493930 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.493980 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.495344 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.495871 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.495877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.495960 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:06.496446 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.496458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.497285 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.497288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.498213 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.498223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.498727 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.498731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.498893 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.499555 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:06.499560 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:06.499610 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:06.499677 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:06.500158 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:06.500166 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:06.500170 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.500331 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.500682 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":126,"threads_started":1}
I20260926 13:54:06.502947 29224 heartbeater.cc:347] Connected to a master server at 127.10.236.254:43623
I20260926 13:54:06.503222 29224 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:06.503827 29224 heartbeater.cc:511] Master 127.10.236.254:43623 requested a full tablet report, sending...
I20260926 13:54:06.504895 28774 ts_manager.cc:194] Registered new tserver with Master: 3b654851f0eb4638b5049202af3e2190 (127.10.236.195:42817)
I20260926 13:54:06.505478 28774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.195:38601
I20260926 13:54:06.506093 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:54:06.512068 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.508248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.10.236.254:43623)
0926 13:54:06.508371 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.508374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.508527 (+ 153us) server_negotiation.cc:407] Connection header received
0926 13:54:06.508598 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.508600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.508641 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.508718 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:06.509068 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.509669 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.510514 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.510725 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.510806 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.511697 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:06.511699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:06.511701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:06.511713 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:06.511732 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:06.511740 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:06.511798 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.511887 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.511934 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:54:06.515311 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:54:06.518352 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:54:06.533116 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.527741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:41479 (local address 127.0.0.1:35572)
0926 13:54:06.528217 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.528236 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.528324 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.529066 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.529072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.529092 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:06.529278 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.529283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.530576 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.530581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.531788 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.531797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.532567 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.532570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.532683 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.532720 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.532834 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.532956 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":104,"thread_start_us":100,"threads_started":1}
I20260926 13:54:06.533121 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.528014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.10.236.194:41479)
0926 13:54:06.528587 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.528600 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.528624 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:54:06.528694 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.528698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.528937 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.529089 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:06.529395 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.530475 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.531909 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.532444 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.532581 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.532765 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.532866 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.532958 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":109,"threads_started":1}
I20260926 13:54:06.533557 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.526994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36947 (local address 127.0.0.1:59408)
0926 13:54:06.527541 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.527593 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.527750 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.529088 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.529091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.529102 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:06.529270 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.529277 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.530328 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.530334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.532034 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.532043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.533196 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.533199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.533297 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.533324 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.533380 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.533430 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":86,"thread_start_us":89,"threads_started":1}
I20260926 13:54:06.533556 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.527195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:42817 (local address 127.0.0.1:46180)
0926 13:54:06.527985 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.527999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.528089 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.528846 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.528852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.528877 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:06.529015 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.529024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.530444 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.530447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.531837 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.531846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.533109 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.533113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.533221 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.533244 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.533313 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.533403 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":125,"thread_start_us":94,"threads_started":1}
I20260926 13:54:06.533905 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.527839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.10.236.195:42817)
0926 13:54:06.528158 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.528172 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.528198 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:54:06.528277 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.528286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.528517 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.528674 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:06.529146 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.530329 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.531972 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.532761 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.532884 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.533047 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.533623 (+ 576us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.533724 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260926 13:54:06.534190 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.527170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.10.236.193:36947)
0926 13:54:06.528583 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.528595 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.528617 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:54:06.528691 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.528698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.528944 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.529421 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:06.529535 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.530221 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.532153 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.532907 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.533010 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.533146 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.533935 (+ 789us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.534041 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1312,"thread_start_us":130,"threads_started":1}
I20260926 13:54:06.536641 29027 tablet_service.cc:1467] Processing CreateTablet for tablet d30e86255a2a4fc9800ec40dfdf59e4d (DEFAULT_TABLE table=test-table [id=65819e15e1bd4920b7e49fcafff9c832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:06.536643 29158 tablet_service.cc:1467] Processing CreateTablet for tablet d30e86255a2a4fc9800ec40dfdf59e4d (DEFAULT_TABLE table=test-table [id=65819e15e1bd4920b7e49fcafff9c832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:06.537786 29158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30e86255a2a4fc9800ec40dfdf59e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:06.537786 29027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30e86255a2a4fc9800ec40dfdf59e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:06.545725 29241 tablet_bootstrap.cc:492] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Bootstrap starting.
I20260926 13:54:06.546969 29242 tablet_bootstrap.cc:492] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Bootstrap starting.
I20260926 13:54:06.546689 28896 tablet_service.cc:1467] Processing CreateTablet for tablet d30e86255a2a4fc9800ec40dfdf59e4d (DEFAULT_TABLE table=test-table [id=65819e15e1bd4920b7e49fcafff9c832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:06.548132 28896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30e86255a2a4fc9800ec40dfdf59e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:06.548759 29241 tablet_bootstrap.cc:654] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:06.549232 29242 tablet_bootstrap.cc:654] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:06.549743 29241 log.cc:824] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:06.549999 29242 log.cc:824] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:06.552664 29242 tablet_bootstrap.cc:492] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: No bootstrap required, opened a new log
I20260926 13:54:06.552876 29242 ts_tablet_manager.cc:1397] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:54:06.553778 29241 tablet_bootstrap.cc:492] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: No bootstrap required, opened a new log
I20260926 13:54:06.553963 29241 ts_tablet_manager.cc:1397] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260926 13:54:06.555038 29245 tablet_bootstrap.cc:492] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: Bootstrap starting.
I20260926 13:54:06.557145 29245 tablet_bootstrap.cc:654] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:06.556953 29242 raft_consensus.cc:348] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.557229 29242 raft_consensus.cc:374] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:06.557296 29242 raft_consensus.cc:729] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393cf783b5e345de9e1deb6e6176bd0e, State: Initialized, Role: FOLLOWER
I20260926 13:54:06.557796 29242 consensus_queue.cc:260] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.558207 29241 raft_consensus.cc:348] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.558444 29241 raft_consensus.cc:374] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:06.558508 29241 raft_consensus.cc:729] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b654851f0eb4638b5049202af3e2190, State: Initialized, Role: FOLLOWER
I20260926 13:54:06.558525 29093 heartbeater.cc:503] Master 127.10.236.254:43623 was elected leader, sending a full tablet report...
I20260926 13:54:06.558830 29242 ts_tablet_manager.cc:1428] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:54:06.558915 29241 consensus_queue.cc:260] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.558373 29245 log.cc:824] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:06.559566 29224 heartbeater.cc:503] Master 127.10.236.254:43623 was elected leader, sending a full tablet report...
I20260926 13:54:06.559922 29241 ts_tablet_manager.cc:1428] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:54:06.561842 29245 tablet_bootstrap.cc:492] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: No bootstrap required, opened a new log
I20260926 13:54:06.562047 29245 ts_tablet_manager.cc:1397] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 13:54:06.566043 29245 raft_consensus.cc:348] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.566294 29245 raft_consensus.cc:374] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:06.566354 29245 raft_consensus.cc:729] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d897bbc4b76b41eb80ef854afdf84544, State: Initialized, Role: FOLLOWER
I20260926 13:54:06.566763 29245 consensus_queue.cc:260] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.567449 28962 heartbeater.cc:503] Master 127.10.236.254:43623 was elected leader, sending a full tablet report...
I20260926 13:54:06.567782 29245 ts_tablet_manager.cc:1428] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260926 13:54:06.578130 28963 tablet.cc:2369] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:54:06.741628 29225 tablet.cc:2369] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:54:06.798833 29094 tablet.cc:2369] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:54:06.823650 29247 raft_consensus.cc:482] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:06.823873 29247 raft_consensus.cc:504] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.825022 29247 leader_election.cc:288] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d897bbc4b76b41eb80ef854afdf84544 (127.10.236.193:36947), 393cf783b5e345de9e1deb6e6176bd0e (127.10.236.194:41479)
I20260926 13:54:06.829478 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.825422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:48907 (local address 127.10.236.193:36947)
0926 13:54:06.825559 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.825566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.825582 (+ 16us) server_negotiation.cc:407] Connection header received
0926 13:54:06.825739 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.825741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.825795 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.825869 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:06.826207 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.826920 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.828257 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.828863 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.828990 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.829040 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.829206 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.829275 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:54:06.829478 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.825323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36947 (local address 127.10.236.195:48907)
0926 13:54:06.825491 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.825508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.825628 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.825901 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.825905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.825928 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:06.826082 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.826090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.827032 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.827035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.828144 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.828153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.828984 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.828988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.829089 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.829117 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.829191 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.829265 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 13:54:06.830008 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.825592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:41479 (local address 127.10.236.195:43993)
0926 13:54:06.825940 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.825953 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:06.826033 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:06.826412 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:06.826415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:06.826433 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:06.826561 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.826566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.827458 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.827461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.828618 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:06.828629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.829552 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.829556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:06.829671 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.829700 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 13:54:06.829783 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:54:06.829853 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":138,"threads_started":1}
I20260926 13:54:06.830024 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.825862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:43993 (local address 127.10.236.194:41479)
0926 13:54:06.826206 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.826211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.826226 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:54:06.826281 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.826284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.826331 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.826402 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:06.826687 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.827351 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.828810 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.829359 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.829545 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.829599 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.829808 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.829908 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260926 13:54:06.830482 28916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30e86255a2a4fc9800ec40dfdf59e4d" candidate_uuid: "3b654851f0eb4638b5049202af3e2190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d897bbc4b76b41eb80ef854afdf84544" is_pre_election: true
I20260926 13:54:06.830741 29047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30e86255a2a4fc9800ec40dfdf59e4d" candidate_uuid: "3b654851f0eb4638b5049202af3e2190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393cf783b5e345de9e1deb6e6176bd0e" is_pre_election: true
I20260926 13:54:06.830782 28916 raft_consensus.cc:2454] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b654851f0eb4638b5049202af3e2190 in term 0.
I20260926 13:54:06.831032 29047 raft_consensus.cc:2454] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b654851f0eb4638b5049202af3e2190 in term 0.
I20260926 13:54:06.831521 29111 leader_election.cc:302] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b654851f0eb4638b5049202af3e2190, d897bbc4b76b41eb80ef854afdf84544; no voters:
I20260926 13:54:06.831804 29247 raft_consensus.cc:2781] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:54:06.831885 29247 raft_consensus.cc:482] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:54:06.831916 29247 raft_consensus.cc:3037] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:06.833021 29247 raft_consensus.cc:504] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.833395 29247 leader_election.cc:288] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [CANDIDATE]: Term 1 election: Requested vote from peers d897bbc4b76b41eb80ef854afdf84544 (127.10.236.193:36947), 393cf783b5e345de9e1deb6e6176bd0e (127.10.236.194:41479)
I20260926 13:54:06.833935 29047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30e86255a2a4fc9800ec40dfdf59e4d" candidate_uuid: "3b654851f0eb4638b5049202af3e2190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393cf783b5e345de9e1deb6e6176bd0e"
I20260926 13:54:06.833946 28916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30e86255a2a4fc9800ec40dfdf59e4d" candidate_uuid: "3b654851f0eb4638b5049202af3e2190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d897bbc4b76b41eb80ef854afdf84544"
I20260926 13:54:06.834055 29047 raft_consensus.cc:3037] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:06.834113 28916 raft_consensus.cc:3037] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:06.835173 29047 raft_consensus.cc:2454] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b654851f0eb4638b5049202af3e2190 in term 1.
I20260926 13:54:06.835201 28916 raft_consensus.cc:2454] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b654851f0eb4638b5049202af3e2190 in term 1.
I20260926 13:54:06.835531 29111 leader_election.cc:302] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393cf783b5e345de9e1deb6e6176bd0e, 3b654851f0eb4638b5049202af3e2190; no voters:
I20260926 13:54:06.835727 29247 raft_consensus.cc:2781] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:06.835995 29247 raft_consensus.cc:686] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 1 LEADER]: Becoming Leader. State: Replica: 3b654851f0eb4638b5049202af3e2190, State: Running, Role: LEADER
I20260926 13:54:06.836376 29247 consensus_queue.cc:237] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } }
I20260926 13:54:06.839775 28773 catalog_manager.cc:5495] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b654851f0eb4638b5049202af3e2190 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "3b654851f0eb4638b5049202af3e2190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b654851f0eb4638b5049202af3e2190" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 42817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 } health_report { overall_health: UNKNOWN } } }
I20260926 13:54:06.905475 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.901892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46182 (local address 127.10.236.195:42817)
0926 13:54:06.902081 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:06.902084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:06.902098 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:54:06.902172 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:06.902175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:06.902226 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:06.902314 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:06.902579 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.903112 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.904006 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:06.904232 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:06.904336 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:06.904627 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:06.905146 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:06.905257 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:06.905315 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 13:54:06.905362 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260926 13:54:06.911886 28916 raft_consensus.cc:1261] T d30e86255a2a4fc9800ec40dfdf59e4d P d897bbc4b76b41eb80ef854afdf84544 [term 1 FOLLOWER]: Refusing update from remote peer 3b654851f0eb4638b5049202af3e2190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:06.911886 29047 raft_consensus.cc:1261] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 1 FOLLOWER]: Refusing update from remote peer 3b654851f0eb4638b5049202af3e2190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:06.912649 29247 consensus_queue.cc:1035] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393cf783b5e345de9e1deb6e6176bd0e" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 41479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:06.912874 29251 consensus_queue.cc:1035] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d897bbc4b76b41eb80ef854afdf84544" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:06.920259 29254 mvcc.cc:205] Tried to move back new op lower bound from 7333604748942483456 to 7333604748644823040. Current Snapshot: MvccSnapshot[applied={T|T < 7333604748942483456}]
I20260926 13:54:06.920871 29256 mvcc.cc:205] Tried to move back new op lower bound from 7333604748942483456 to 7333604748644823040. Current Snapshot: MvccSnapshot[applied={T|T < 7333604748942483456}]
I20260926 13:54:06.929188 28773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:06.929361 28773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:06.931728 28773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c57e0ec874154a3aad8482c80723e869: Sending DeleteTablet for 3 replicas of tablet d30e86255a2a4fc9800ec40dfdf59e4d
I20260926 13:54:06.932538 29158 tablet_service.cc:1514] Processing DeleteTablet for tablet d30e86255a2a4fc9800ec40dfdf59e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:06 UTC) from {username='slave'} at 127.0.0.1:46180
I20260926 13:54:06.932651 28896 tablet_service.cc:1514] Processing DeleteTablet for tablet d30e86255a2a4fc9800ec40dfdf59e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:06 UTC) from {username='slave'} at 127.0.0.1:59408
I20260926 13:54:06.932753 29027 tablet_service.cc:1514] Processing DeleteTablet for tablet d30e86255a2a4fc9800ec40dfdf59e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:06 UTC) from {username='slave'} at 127.0.0.1:35572
I20260926 13:54:06.933254 29265 tablet_replica.cc:330] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: stopping tablet replica
I20260926 13:54:06.933586 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28834
I20260926 13:54:06.933588 29265 raft_consensus.cc:2229] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:06.933892 29265 raft_consensus.cc:2258] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:06.935679 29265 ts_tablet_manager.cc:1905] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:06.938462 29265 ts_tablet_manager.cc:1918] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:06.938560 29265 log.cc:1196] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/d30e86255a2a4fc9800ec40dfdf59e4d
I20260926 13:54:06.938866 29265 ts_tablet_manager.cc:1939] T d30e86255a2a4fc9800ec40dfdf59e4d P 393cf783b5e345de9e1deb6e6176bd0e: Deleting consensus metadata
I20260926 13:54:06.938860 29263 tablet_replica.cc:330] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: stopping tablet replica
I20260926 13:54:06.939265 29263 raft_consensus.cc:2229] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:54:06.939777 29263 raft_consensus.cc:2258] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:06.940390 28758 catalog_manager.cc:4841] TS 393cf783b5e345de9e1deb6e6176bd0e (127.10.236.194:41479): tablet d30e86255a2a4fc9800ec40dfdf59e4d (table test-table [id=65819e15e1bd4920b7e49fcafff9c832]) successfully deleted
I20260926 13:54:06.941548 29263 ts_tablet_manager.cc:1905] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:54:06.942960 28758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:36947 (error 108)
I20260926 13:54:06.943009 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28965
I20260926 13:54:06.944891 29263 ts_tablet_manager.cc:1918] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:06.944960 29263 log.cc:1196] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/d30e86255a2a4fc9800ec40dfdf59e4d
I20260926 13:54:06.945181 29263 ts_tablet_manager.cc:1939] T d30e86255a2a4fc9800ec40dfdf59e4d P 3b654851f0eb4638b5049202af3e2190: Deleting consensus metadata
I20260926 13:54:06.946211 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:06.945885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:36947 (local address 127.0.0.1:59416)
0926 13:54:06.946046 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:06.946121 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:36947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 13:54:06.946425 28760 catalog_manager.cc:4841] TS 3b654851f0eb4638b5049202af3e2190 (127.10.236.195:42817): tablet d30e86255a2a4fc9800ec40dfdf59e4d (table test-table [id=65819e15e1bd4920b7e49fcafff9c832]) successfully deleted
W20260926 13:54:06.946446 28758 catalog_manager.cc:4584] TS d897bbc4b76b41eb80ef854afdf84544 (127.10.236.193:36947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d30e86255a2a4fc9800ec40dfdf59e4d: Network error: Client connection negotiation failed: client connection to 127.10.236.193:36947: connect: Connection refused (error 111)
I20260926 13:54:06.950321 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29096
I20260926 13:54:06.956604 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 28743
2026-09-26T13:54:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1368 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:07 dist-test-slave-r34j krb5kdc[29269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:07 dist-test-slave-r34j krb5kdc[29269](info): set up 2 sockets
Sep 26 13:54:07 dist-test-slave-r34j krb5kdc[29269](info): commencing operation
krb5kdc: starting...
W20260926 13:54:08.035392 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:08.176061 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33043
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:33043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:08.290097 29282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:08.290393 29282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:08.290436 29282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:08.294682 29282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:08.294756 29282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:08.294776 29282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:08.294793 29282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:08.294808 29282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:08.300259 29282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:33043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:08.301309 29282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:08.302312 29282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:08.311128 29287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:08.311149 29288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:08.311264 29282 server_base.cc:1034] running on GCE node
W20260926 13:54:08.311115 29290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:08.311981 29282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:08.313134 29282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:08.314322 29282 hybrid_clock.cc:648] HybridClock initialized: now 1790430848314301 us; error 37 us; skew 500 ppm
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:08.322628 29282 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:08.323701 29282 webserver.cc:460] Webserver started at http://127.10.236.254:40865/ using document root <none> and password file <none>
I20260926 13:54:08.324283 29282 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:08.324381 29282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:08.324615 29282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:08.326812 29282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "12e01025eaea4da3a72f32f0eec5ad25"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "425f96a795ab89deb48f717c50256f29"
server_key_iv: "dfa56e4ae782be4257bdc30a964d72a5"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.327386 29282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "12e01025eaea4da3a72f32f0eec5ad25"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "425f96a795ab89deb48f717c50256f29"
server_key_iv: "dfa56e4ae782be4257bdc30a964d72a5"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.331302 29282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 13:54:08.334246 29297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.335628 29282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:08.335765 29282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "12e01025eaea4da3a72f32f0eec5ad25"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "425f96a795ab89deb48f717c50256f29"
server_key_iv: "dfa56e4ae782be4257bdc30a964d72a5"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.335886 29282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:08.374434 29282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:08.378006 29282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:08.378224 29282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:08.385458 29282 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:33043
I20260926 13:54:08.385485 29349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:33043 every 8 connection(s)
I20260926 13:54:08.386425 29282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:08.389318 29350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:08.393179 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29282
I20260926 13:54:08.393296 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:08.393599 11187 external_mini_cluster.cc:1428] Setting key 6875bc8dbf81a3f49ea55b567a0f4503
I20260926 13:54:08.395146 29350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: Bootstrap starting.
I20260926 13:54:08.397576 29350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:08.398391 29350 log.cc:824] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:08.400568 29350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: No bootstrap required, opened a new log
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:08.404806 29350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e01025eaea4da3a72f32f0eec5ad25" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33043 } }
I20260926 13:54:08.405054 29350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:08.405133 29350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e01025eaea4da3a72f32f0eec5ad25, State: Initialized, Role: FOLLOWER
I20260926 13:54:08.405575 29350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e01025eaea4da3a72f32f0eec5ad25" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33043 } }
I20260926 13:54:08.405730 29350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:08.405812 29350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:08.405923 29350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:08.406821 29350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e01025eaea4da3a72f32f0eec5ad25" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33043 } }
I20260926 13:54:08.407164 29350 leader_election.cc:302] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e01025eaea4da3a72f32f0eec5ad25; no voters:
I20260926 13:54:08.407511 29350 leader_election.cc:288] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:08.407588 29355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:08.407778 29355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [term 1 LEADER]: Becoming Leader. State: Replica: 12e01025eaea4da3a72f32f0eec5ad25, State: Running, Role: LEADER
I20260926 13:54:08.408280 29355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e01025eaea4da3a72f32f0eec5ad25" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33043 } }
I20260926 13:54:08.408877 29350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:08.409375 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:08.395095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.10.236.254:33043)
0926 13:54:08.395593 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:08.395612 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:08.395650 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:54:08.396382 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:08.396400 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:08.396759 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:08.397075 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:08.398038 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.399266 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.400465 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.400993 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.401119 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:08.403946 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:08.403951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:08.403965 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:08.403992 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:08.406025 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:08.406452 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:08.406457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:08.406463 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:08.406580 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:08.406868 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:08.406872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:08.406874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:08.407365 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:08.407594 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:08.408215 (+ 621us) server_negotiation.cc:299] Negotiation successful
0926 13:54:08.408516 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":170,"threads_started":1}
I20260926 13:54:08.409480 29355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e01025eaea4da3a72f32f0eec5ad25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e01025eaea4da3a72f32f0eec5ad25" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33043 } } }
I20260926 13:54:08.410275 29355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:08.410167 29356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e01025eaea4da3a72f32f0eec5ad25. Latest consensus state: current_term: 1 leader_uuid: "12e01025eaea4da3a72f32f0eec5ad25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e01025eaea4da3a72f32f0eec5ad25" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33043 } } }
I20260926 13:54:08.411813 29356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25 [sys.catalog]: This master's current role is: LEADER
W20260926 13:54:08.414330 29369 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:08.414429 29369 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:08.414530 29371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:08.415268 29371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:08.421238 29371 catalog_manager.cc:1347] Generated new cluster ID: 17e8a51d5eba473db3046f1a5698b4f9
I20260926 13:54:08.421327 29371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:08.431828 29371 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:08.433163 29371 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:08.447875 29371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: Generated new TSK 0
I20260926 13:54:08.448685 29371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:54:08.492200 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33043
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:08.611862 29376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:08.612174 29376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:08.612243 29376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:08.616302 29376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:08.616390 29376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:08.616480 29376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:54:08.622035 29376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:08.623179 29376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:08.624243 29376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:54:08.634562 29376 server_base.cc:1034] running on GCE node
W20260926 13:54:08.634613 29381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:08.634598 29382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:08.634743 29384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:08.635146 29376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:08.635829 29376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:08.637056 29376 hybrid_clock.cc:648] HybridClock initialized: now 1790430848637016 us; error 46 us; skew 500 ppm
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:08.640041 29376 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:54:08.641008 29376 webserver.cc:460] Webserver started at http://127.10.236.193:35111/ using document root <none> and password file <none>
I20260926 13:54:08.641613 29376 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:08.641674 29376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:08.641885 29376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:08.644110 29376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "4c59a685d57c4e1d9fbf5b993b5b787f"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "ffc7018241b87a245beea4c398958753"
server_key_iv: "69cc5fc029eee0ce766c470b17756dc2"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.644665 29376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "4c59a685d57c4e1d9fbf5b993b5b787f"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "ffc7018241b87a245beea4c398958753"
server_key_iv: "69cc5fc029eee0ce766c470b17756dc2"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.648592 29376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:54:08.651067 29391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.652514 29376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:08.652638 29376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "4c59a685d57c4e1d9fbf5b993b5b787f"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "ffc7018241b87a245beea4c398958753"
server_key_iv: "69cc5fc029eee0ce766c470b17756dc2"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.652750 29376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:08.692469 29376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:08.695832 29376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:08.696059 29376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:08.696671 29376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:08.697530 29376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:08.697582 29376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.697659 29376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:08.697688 29376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.707101 29376 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:40791
I20260926 13:54:08.707149 29504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:40791 every 8 connection(s)
I20260926 13:54:08.708117 29376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:54:08.709728 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29376
I20260926 13:54:08.709918 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:54:08.710301 11187 external_mini_cluster.cc:1428] Setting key d5ed2ba86b92500e71c48ee9b2bfad79
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
I20260926 13:54:08.722903 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:08.710048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:56375 (local address 127.10.236.254:33043)
0926 13:54:08.710222 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:08.710229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:08.711187 (+ 958us) server_negotiation.cc:407] Connection header received
0926 13:54:08.712655 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:08.712658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:08.712715 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:08.712791 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:08.713968 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.714518 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.715754 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.715986 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.716100 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:08.718791 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:08.718794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:08.718796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:08.718819 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:08.720410 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:08.720936 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:08.720940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:08.720942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:08.721026 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:08.721462 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:08.721467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:08.721469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:08.721667 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:08.721773 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:08.722452 (+ 679us) server_negotiation.cc:299] Negotiation successful
0926 13:54:08.722619 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:54:08.723611 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:08.710530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33043 (local address 127.10.236.193:56375)
0926 13:54:08.711039 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:08.711092 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:08.712437 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:08.712999 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:08.713005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:08.713349 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:08.713824 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:08.713837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.714652 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.714655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:08.715628 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:08.715637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.716133 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.716136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:08.716310 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:08.716913 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:08.716928 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:08.718614 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:08.720542 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:08.720548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:08.720564 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:08.720832 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:08.721173 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:08.721176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:08.721178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:08.721367 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:08.721771 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:08.721778 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:08.722124 (+ 346us) client_negotiation.cc:769] Sending connection context
0926 13:54:08.722343 (+ 219us) client_negotiation.cc:240] Negotiation successful
0926 13:54:08.722639 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":135,"threads_started":1}
I20260926 13:54:08.725000 29505 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33043
I20260926 13:54:08.725373 29505 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:08.725868 29505 heartbeater.cc:511] Master 127.10.236.254:33043 requested a full tablet report, sending...
I20260926 13:54:08.727458 29314 ts_manager.cc:194] Registered new tserver with Master: 4c59a685d57c4e1d9fbf5b993b5b787f (127.10.236.193:40791)
I20260926 13:54:08.728683 29314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:56375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:54:08.740010 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33043
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:08.850651 29510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:08.850935 29510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:08.851011 29510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:08.854988 29510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:08.855093 29510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:08.855182 29510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:54:08.860575 29510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:08.861670 29510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:08.862713 29510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:08.871942 29515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:08.872123 29518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:08.872159 29516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:08.872200 29510 server_base.cc:1034] running on GCE node
I20260926 13:54:08.872802 29510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:08.873456 29510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:08.874660 29510 hybrid_clock.cc:648] HybridClock initialized: now 1790430848874629 us; error 35 us; skew 500 ppm
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:08.877594 29510 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:54:08.878613 29510 webserver.cc:460] Webserver started at http://127.10.236.194:41553/ using document root <none> and password file <none>
I20260926 13:54:08.879163 29510 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:08.879223 29510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:08.879390 29510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:08.881590 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "2524626c19b342d598d1257465d88dc4"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "b634bdb41f8eb230e4c9009fc8b1a3ef"
server_key_iv: "210b3905eb3d59b30b670361a803d8d2"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.882112 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "2524626c19b342d598d1257465d88dc4"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "b634bdb41f8eb230e4c9009fc8b1a3ef"
server_key_iv: "210b3905eb3d59b30b670361a803d8d2"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.886039 29510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:08.888818 29525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.890288 29510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:54:08.890398 29510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "2524626c19b342d598d1257465d88dc4"
format_stamp: "Formatted at 2026-09-26 13:54:08 on dist-test-slave-r34j"
server_key: "b634bdb41f8eb230e4c9009fc8b1a3ef"
server_key_iv: "210b3905eb3d59b30b670361a803d8d2"
server_key_version: "encryptionkey@0"
I20260926 13:54:08.890578 29510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:08.906507 29510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:08.910200 29510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:08.910449 29510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:08.911161 29510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:08.912150 29510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:08.912222 29510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.912297 29510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:08.912348 29510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:08.922372 29510 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:39853
I20260926 13:54:08.922401 29638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:39853 every 8 connection(s)
I20260926 13:54:08.923389 29510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:54:08.926738 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29510
I20260926 13:54:08.926885 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:54:08.927242 11187 external_mini_cluster.cc:1428] Setting key 9c1e979e35a4981acee32ab5e29b89c5
Sep 26 13:54:08 dist-test-slave-r34j krb5kdc[29269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430848, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
I20260926 13:54:08.938736 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:08.925367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:53141 (local address 127.10.236.254:33043)
0926 13:54:08.925531 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:08.925535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:08.926365 (+ 830us) server_negotiation.cc:407] Connection header received
0926 13:54:08.928115 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:08.928118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:08.928165 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:08.928258 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:08.929813 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.930406 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.931617 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.931837 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.931919 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:08.934752 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:08.934757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:08.934760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:08.934792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:08.936475 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:08.937025 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:08.937028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:08.937030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:08.937117 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:08.937554 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:08.937561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:08.937565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:08.937768 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:08.937849 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:08.938465 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 13:54:08.938598 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 13:54:08.939775 29641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:08.925702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33043 (local address 127.10.236.194:53141)
0926 13:54:08.926201 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:08.926244 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:08.927874 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:08.928408 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:08.928415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:08.928900 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:08.929655 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:08.929679 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.930551 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.930554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:08.931480 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:08.931515 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:08.931997 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:08.932004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:08.932236 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:08.932875 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:08.932903 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:08.934581 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:08.936648 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:08.936653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:08.936665 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:08.936926 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:08.937231 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:08.937234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:08.937236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:08.937450 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:08.937887 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:08.937893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:08.938190 (+ 297us) client_negotiation.cc:769] Sending connection context
0926 13:54:08.938416 (+ 226us) client_negotiation.cc:240] Negotiation successful
0926 13:54:08.938720 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":129,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:54:08.941227 29639 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33043
I20260926 13:54:08.941609 29639 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:08.942260 29639 heartbeater.cc:511] Master 127.10.236.254:33043 requested a full tablet report, sending...
I20260926 13:54:08.943619 29314 ts_manager.cc:194] Registered new tserver with Master: 2524626c19b342d598d1257465d88dc4 (127.10.236.194:39853)
I20260926 13:54:08.944242 29314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:53141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:54:08.959383 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33043
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:09.073201 29644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:09.073529 29644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:09.073607 29644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:09.077646 29644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:09.077749 29644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:09.077841 29644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:54:09.083380 29644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:09.084493 29644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:09.085538 29644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:09.095330 29650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:09.095347 29652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.095625 29644 server_base.cc:1034] running on GCE node
W20260926 13:54:09.095381 29649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.096138 29644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:09.096719 29644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:09.097885 29644 hybrid_clock.cc:648] HybridClock initialized: now 1790430849097862 us; error 37 us; skew 500 ppm
Sep 26 13:54:09 dist-test-slave-r34j krb5kdc[29269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430849, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:09.100749 29644 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:54:09.101550 29644 webserver.cc:460] Webserver started at http://127.10.236.195:34669/ using document root <none> and password file <none>
I20260926 13:54:09.102053 29644 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:09.102108 29644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:09.102264 29644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:09.104286 29644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "8f1c5963f14748fea5b49a646f70e67e"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "383d7415b895ce4536485debc83f9bfb"
server_key_iv: "cdbe425d6b277a06db10e502332fd9e4"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.104787 29644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "8f1c5963f14748fea5b49a646f70e67e"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "383d7415b895ce4536485debc83f9bfb"
server_key_iv: "cdbe425d6b277a06db10e502332fd9e4"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.108407 29644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:54:09.111083 29659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.112370 29644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:54:09.112497 29644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "8f1c5963f14748fea5b49a646f70e67e"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "383d7415b895ce4536485debc83f9bfb"
server_key_iv: "cdbe425d6b277a06db10e502332fd9e4"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.112625 29644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:09.128273 29644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:09.131603 29644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:09.131789 29644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:09.132354 29644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:09.133162 29644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:09.133229 29644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.133294 29644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:09.133340 29644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.142423 29644 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:35437
I20260926 13:54:09.142454 29772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:35437 every 8 connection(s)
I20260926 13:54:09.143317 29644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:54:09.146183 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29644
I20260926 13:54:09.146328 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:54:09.146647 11187 external_mini_cluster.cc:1428] Setting key 12175e3f92bfe46f1c6277c1e215b1d1
Sep 26 13:54:09 dist-test-slave-r34j krb5kdc[29269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430849, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:09.157703 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.145332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:51187 (local address 127.10.236.254:33043)
0926 13:54:09.145490 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.145496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.146433 (+ 937us) server_negotiation.cc:407] Connection header received
0926 13:54:09.148115 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.148119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.148176 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.148252 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:09.149507 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.150050 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.151173 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.151402 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.151531 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.154070 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:09.154073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:09.154074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:09.154100 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:09.155563 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:09.156069 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:09.156072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:09.156075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:09.156158 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:09.156489 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:09.156493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:09.156495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:09.156753 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:09.156852 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.157396 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.157527 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:54:09.158589 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.145738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33043 (local address 127.10.236.195:51187)
0926 13:54:09.146282 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.146340 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.147896 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.148419 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.148426 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.148878 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:09.149363 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.149376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.150170 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.150173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.151051 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.151060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.151570 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.151574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.151740 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.152329 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:09.152346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:09.153889 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:09.155673 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:09.155679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:09.155695 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:09.155962 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:09.156254 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:09.156257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:09.156259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:09.156397 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:09.156860 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:09.156868 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:09.157132 (+ 264us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.157355 (+ 223us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.157733 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":161,"threads_started":1}
I20260926 13:54:09.159852 29773 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33043
I20260926 13:54:09.160100 29773 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:09.160588 29773 heartbeater.cc:511] Master 127.10.236.254:33043 requested a full tablet report, sending...
I20260926 13:54:09.161544 29314 ts_manager.cc:194] Registered new tserver with Master: 8f1c5963f14748fea5b49a646f70e67e (127.10.236.195:35437)
I20260926 13:54:09.162170 29314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:51187
I20260926 13:54:09.172370 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:54:09.182633 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.174427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.10.236.254:33043)
0926 13:54:09.174609 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.174614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.174762 (+ 148us) server_negotiation.cc:407] Connection header received
0926 13:54:09.174929 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.174932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.174978 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.175058 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:09.175882 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.176408 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.177260 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.177518 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.177620 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.179314 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:09.179318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:09.179321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:09.179347 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:09.180725 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:09.181231 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:09.181235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:09.181236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:09.181320 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:09.181640 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:09.181643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:09.181645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:09.181859 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:09.182017 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.182320 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.182436 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 13:54:09.185678 29314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:54:09.188789 29314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:54:09.203459 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.197220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35437 (local address 127.0.0.1:57862)
0926 13:54:09.197567 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.197621 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.197752 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.198850 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.198855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.198877 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:09.199020 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.199033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.200561 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.200564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.201741 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.201750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.202873 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.202878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.202982 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.203073 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.203149 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.203298 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":155,"thread_start_us":105,"threads_started":1}
I20260926 13:54:09.203523 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.197407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.10.236.195:35437)
0926 13:54:09.198349 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.198362 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.198385 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:54:09.198461 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.198467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.198685 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.198861 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:09.199138 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.200423 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.201862 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.202722 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.203082 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.203206 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.203257 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.203347 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":137,"threads_started":1}
I20260926 13:54:09.203562 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.198021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.10.236.194:39853)
0926 13:54:09.198739 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.198752 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.198774 (+ 22us) server_negotiation.cc:407] Connection header received
0926 13:54:09.198845 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.198850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.199048 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.199229 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:09.199660 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.200750 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.202169 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.202714 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.202849 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.202987 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.203298 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.203390 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":141,"threads_started":1}
I20260926 13:54:09.203708 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.198424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.10.236.193:40791)
0926 13:54:09.198739 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.198753 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.198774 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:54:09.198845 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.198850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.199041 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.199229 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:09.199446 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.200270 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.202026 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.202806 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.203295 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.203402 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.203445 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.203570 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":130,"threads_started":1}
I20260926 13:54:09.204505 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.197955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40791 (local address 127.0.0.1:37288)
0926 13:54:09.198390 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.198403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.198485 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.199221 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.199224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.199243 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:09.199330 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.199336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.200396 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.200402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.201902 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.201913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.202918 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.202921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.203004 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.203046 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.203134 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.204374 (+ 1240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":1137,"thread_start_us":95,"threads_started":1}
I20260926 13:54:09.205441 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.197591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:39853 (local address 127.0.0.1:60714)
0926 13:54:09.197992 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.198004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.198098 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.199382 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.199404 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.199429 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:09.199538 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.199546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.200895 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.200899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.202062 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.202069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.202838 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.202842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.202956 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.203057 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.203154 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.205321 (+ 2167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":2103,"thread_start_us":106,"threads_started":1}
I20260926 13:54:09.206501 29707 tablet_service.cc:1467] Processing CreateTablet for tablet 642c883889624c0c915a649167d13050 (DEFAULT_TABLE table=test-table [id=4850fb4906e345c8a528caf3a22ff3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:09.207471 29707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642c883889624c0c915a649167d13050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:09.207245 29439 tablet_service.cc:1467] Processing CreateTablet for tablet 642c883889624c0c915a649167d13050 (DEFAULT_TABLE table=test-table [id=4850fb4906e345c8a528caf3a22ff3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:09.208314 29439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642c883889624c0c915a649167d13050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:09.208041 29573 tablet_service.cc:1467] Processing CreateTablet for tablet 642c883889624c0c915a649167d13050 (DEFAULT_TABLE table=test-table [id=4850fb4906e345c8a528caf3a22ff3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:09.209014 29573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642c883889624c0c915a649167d13050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:09.213654 29790 tablet_bootstrap.cc:492] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Bootstrap starting.
I20260926 13:54:09.215255 29791 tablet_bootstrap.cc:492] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: Bootstrap starting.
I20260926 13:54:09.216034 29790 tablet_bootstrap.cc:654] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:09.216673 29792 tablet_bootstrap.cc:492] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Bootstrap starting.
I20260926 13:54:09.217494 29790 log.cc:824] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:09.217711 29791 tablet_bootstrap.cc:654] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:09.218617 29792 tablet_bootstrap.cc:654] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:09.218876 29791 log.cc:824] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:09.219632 29790 tablet_bootstrap.cc:492] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: No bootstrap required, opened a new log
I20260926 13:54:09.219822 29790 ts_tablet_manager.cc:1397] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260926 13:54:09.219825 29792 log.cc:824] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:09.220800 29791 tablet_bootstrap.cc:492] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: No bootstrap required, opened a new log
I20260926 13:54:09.221035 29791 ts_tablet_manager.cc:1397] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:09.223768 29790 raft_consensus.cc:348] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.224046 29790 raft_consensus.cc:374] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:09.224128 29790 raft_consensus.cc:729] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1c5963f14748fea5b49a646f70e67e, State: Initialized, Role: FOLLOWER
I20260926 13:54:09.224586 29790 consensus_queue.cc:260] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.224766 29791 raft_consensus.cc:348] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.225059 29791 raft_consensus.cc:374] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:09.225153 29791 raft_consensus.cc:729] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c59a685d57c4e1d9fbf5b993b5b787f, State: Initialized, Role: FOLLOWER
I20260926 13:54:09.225208 29792 tablet_bootstrap.cc:492] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: No bootstrap required, opened a new log
I20260926 13:54:09.225261 29773 heartbeater.cc:503] Master 127.10.236.254:33043 was elected leader, sending a full tablet report...
I20260926 13:54:09.225431 29792 ts_tablet_manager.cc:1397] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260926 13:54:09.225606 29790 ts_tablet_manager.cc:1428] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:09.225647 29791 consensus_queue.cc:260] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.226282 29505 heartbeater.cc:503] Master 127.10.236.254:33043 was elected leader, sending a full tablet report...
I20260926 13:54:09.226603 29791 ts_tablet_manager.cc:1428] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:09.229411 29792 raft_consensus.cc:348] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.229713 29792 raft_consensus.cc:374] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:09.229799 29792 raft_consensus.cc:729] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2524626c19b342d598d1257465d88dc4, State: Initialized, Role: FOLLOWER
I20260926 13:54:09.230247 29792 consensus_queue.cc:260] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.231063 29639 heartbeater.cc:503] Master 127.10.236.254:33043 was elected leader, sending a full tablet report...
I20260926 13:54:09.231271 29792 ts_tablet_manager.cc:1428] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:09.236459 29798 raft_consensus.cc:482] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:09.236609 29798 raft_consensus.cc:504] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.237615 29798 leader_election.cc:288] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f1c5963f14748fea5b49a646f70e67e (127.10.236.195:35437), 4c59a685d57c4e1d9fbf5b993b5b787f (127.10.236.193:40791)
I20260926 13:54:09.241425 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.237908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:35387 (local address 127.10.236.195:35437)
0926 13:54:09.238069 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.238074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.238089 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:54:09.238233 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.238239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.238299 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.238380 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:09.238733 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.239310 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.240491 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.240891 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.241014 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.241076 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.241239 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.241303 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:54:09.241425 29641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.237848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35437 (local address 127.10.236.194:35387)
0926 13:54:09.237986 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.238005 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.238107 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.238501 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.238503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.238529 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:09.238614 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.238618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.239422 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.239425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.240375 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.240384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.241004 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.241008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.241116 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.241141 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.241224 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.241298 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":5}
I20260926 13:54:09.242095 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642c883889624c0c915a649167d13050" candidate_uuid: "2524626c19b342d598d1257465d88dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f1c5963f14748fea5b49a646f70e67e" is_pre_election: true
I20260926 13:54:09.242394 29727 raft_consensus.cc:2454] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2524626c19b342d598d1257465d88dc4 in term 0.
I20260926 13:54:09.242457 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.238144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40791 (local address 127.10.236.194:57361)
0926 13:54:09.238493 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.238504 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.238617 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.238875 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.238878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.238895 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:09.239028 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.239033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.239929 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.239936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.240944 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.240954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.242000 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.242004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.242118 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.242143 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.242241 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.242310 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
I20260926 13:54:09.242444 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.238245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:57361 (local address 127.10.236.193:40791)
0926 13:54:09.238363 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.238367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.238580 (+ 213us) server_negotiation.cc:407] Connection header received
0926 13:54:09.238709 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.238714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.238766 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.238852 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:09.239156 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.239832 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.241071 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.241544 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.241672 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.241733 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.242258 (+ 525us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.242331 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:54:09.242929 29526 leader_election.cc:302] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2524626c19b342d598d1257465d88dc4, 8f1c5963f14748fea5b49a646f70e67e; no voters:
I20260926 13:54:09.243194 29798 raft_consensus.cc:2781] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:54:09.243311 29798 raft_consensus.cc:482] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:54:09.243376 29798 raft_consensus.cc:3037] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:09.243621 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642c883889624c0c915a649167d13050" candidate_uuid: "2524626c19b342d598d1257465d88dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" is_pre_election: true
I20260926 13:54:09.243827 29459 raft_consensus.cc:2454] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2524626c19b342d598d1257465d88dc4 in term 0.
I20260926 13:54:09.244616 29798 raft_consensus.cc:504] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.244913 29798 leader_election.cc:288] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [CANDIDATE]: Term 1 election: Requested vote from peers 8f1c5963f14748fea5b49a646f70e67e (127.10.236.195:35437), 4c59a685d57c4e1d9fbf5b993b5b787f (127.10.236.193:40791)
I20260926 13:54:09.245270 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642c883889624c0c915a649167d13050" candidate_uuid: "2524626c19b342d598d1257465d88dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f1c5963f14748fea5b49a646f70e67e"
I20260926 13:54:09.245349 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642c883889624c0c915a649167d13050" candidate_uuid: "2524626c19b342d598d1257465d88dc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f"
I20260926 13:54:09.245406 29727 raft_consensus.cc:3037] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:09.245486 29459 raft_consensus.cc:3037] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:09.246773 29727 raft_consensus.cc:2454] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2524626c19b342d598d1257465d88dc4 in term 1.
I20260926 13:54:09.246773 29459 raft_consensus.cc:2454] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2524626c19b342d598d1257465d88dc4 in term 1.
I20260926 13:54:09.247131 29526 leader_election.cc:302] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2524626c19b342d598d1257465d88dc4, 8f1c5963f14748fea5b49a646f70e67e; no voters:
I20260926 13:54:09.247344 29798 raft_consensus.cc:2781] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:09.247620 29798 raft_consensus.cc:686] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 1 LEADER]: Becoming Leader. State: Replica: 2524626c19b342d598d1257465d88dc4, State: Running, Role: LEADER
I20260926 13:54:09.247960 29798 consensus_queue.cc:237] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } }
I20260926 13:54:09.251084 29314 catalog_manager.cc:5495] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2524626c19b342d598d1257465d88dc4 (127.10.236.194). New cstate: current_term: 1 leader_uuid: "2524626c19b342d598d1257465d88dc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2524626c19b342d598d1257465d88dc4" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 39853 } health_report { overall_health: HEALTHY } } }
I20260926 13:54:09.274560 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.270683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.10.236.194:39853)
0926 13:54:09.270813 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.270816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.270830 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:54:09.270931 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.270933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.270986 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.271111 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:09.271354 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.271862 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.272760 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.273048 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.273167 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.273455 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:09.273871 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:09.274033 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.274388 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.274428 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:54:09.281028 29727 raft_consensus.cc:1261] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 1 FOLLOWER]: Refusing update from remote peer 2524626c19b342d598d1257465d88dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:09.281045 29459 raft_consensus.cc:1261] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 1 FOLLOWER]: Refusing update from remote peer 2524626c19b342d598d1257465d88dc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:09.281749 29800 consensus_queue.cc:1035] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c59a685d57c4e1d9fbf5b993b5b787f" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:09.281908 29798 consensus_queue.cc:1035] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1c5963f14748fea5b49a646f70e67e" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:09.289422 29805 mvcc.cc:205] Tried to move back new op lower bound from 7333604758646562816 to 7333604758522601472. Current Snapshot: MvccSnapshot[applied={T|T < 7333604758646562816}]
I20260926 13:54:09.289530 29804 mvcc.cc:205] Tried to move back new op lower bound from 7333604758646562816 to 7333604758522601472. Current Snapshot: MvccSnapshot[applied={T|T < 7333604758646562816}]
I20260926 13:54:09.301306 29313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:09.301505 29313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:09.304196 29313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12e01025eaea4da3a72f32f0eec5ad25: Sending DeleteTablet for 3 replicas of tablet 642c883889624c0c915a649167d13050
I20260926 13:54:09.305094 29439 tablet_service.cc:1514] Processing DeleteTablet for tablet 642c883889624c0c915a649167d13050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:09 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:37288
I20260926 13:54:09.305095 29573 tablet_service.cc:1514] Processing DeleteTablet for tablet 642c883889624c0c915a649167d13050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:09 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:60714
I20260926 13:54:09.305125 29707 tablet_service.cc:1514] Processing DeleteTablet for tablet 642c883889624c0c915a649167d13050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:09 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:57862
I20260926 13:54:09.305625 29815 tablet_replica.cc:330] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f: stopping tablet replica
I20260926 13:54:09.305956 29815 raft_consensus.cc:2229] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:09.306274 29814 tablet_replica.cc:330] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: stopping tablet replica
I20260926 13:54:09.306381 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29376
I20260926 13:54:09.306414 29815 raft_consensus.cc:2258] T 642c883889624c0c915a649167d13050 P 4c59a685d57c4e1d9fbf5b993b5b787f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:09.308053 29813 tablet_replica.cc:330] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: stopping tablet replica
I20260926 13:54:09.308120 29814 raft_consensus.cc:2229] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:09.308416 29814 raft_consensus.cc:2258] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:09.308629 29813 raft_consensus.cc:2229] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:54:09.309106 29813 raft_consensus.cc:2258] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:09.310457 29814 ts_tablet_manager.cc:1905] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:09.310631 29813 ts_tablet_manager.cc:1905] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:09.313335 29814 ts_tablet_manager.cc:1918] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:09.313515 29814 log.cc:1196] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/642c883889624c0c915a649167d13050
I20260926 13:54:09.313920 29814 ts_tablet_manager.cc:1939] T 642c883889624c0c915a649167d13050 P 8f1c5963f14748fea5b49a646f70e67e: Deleting consensus metadata
I20260926 13:54:09.314133 29813 ts_tablet_manager.cc:1918] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:09.314301 29813 log.cc:1196] T 642c883889624c0c915a649167d13050 P 2524626c19b342d598d1257465d88dc4: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/642c883889624c0c915a649167d13050
W20260926 13:54:09.314862 29298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:40791 (error 108)
I20260926 13:54:09.314934 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29510
I20260926 13:54:09.316049 29298 catalog_manager.cc:4841] TS 8f1c5963f14748fea5b49a646f70e67e (127.10.236.195:35437): tablet 642c883889624c0c915a649167d13050 (table test-table [id=4850fb4906e345c8a528caf3a22ff3c1]) successfully deleted
I20260926 13:54:09.317233 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.316928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40791 (local address 127.0.0.1:37292)
0926 13:54:09.317054 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.317162 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:40791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 13:54:09.317380 29298 catalog_manager.cc:4584] TS 4c59a685d57c4e1d9fbf5b993b5b787f (127.10.236.193:40791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 642c883889624c0c915a649167d13050: Network error: Client connection negotiation failed: client connection to 127.10.236.193:40791: connect: Connection refused (error 111)
I20260926 13:54:09.321373 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29644
I20260926 13:54:09.322825 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.321554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:39853 (local address 127.0.0.1:60728)
0926 13:54:09.322657 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.322725 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.194:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":987}
I20260926 13:54:09.326822 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29282
2026-09-26T13:54:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2370 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T13:54:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:09Z Disabled control of system clock
I20260926 13:54:09.357075 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:40803
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:40803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:54:09.466900 29820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:09.467207 29820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:09.467247 29820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:09.471262 29820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:09.471325 29820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:09.471341 29820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:09.471357 29820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:09.471372 29820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:09.476121 29820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:40803
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:40803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:09.477180 29820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:09.478545 29820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:09.487416 29826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.487586 29820 server_base.cc:1034] running on GCE node
W20260926 13:54:09.487434 29825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:09.487422 29828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.488336 29820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:09.489457 29820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:09.490656 29820 hybrid_clock.cc:648] HybridClock initialized: now 1790430849490638 us; error 32 us; skew 500 ppm
I20260926 13:54:09.492727 29820 webserver.cc:460] Webserver started at http://127.10.236.254:41103/ using document root <none> and password file <none>
I20260926 13:54:09.493315 29820 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:09.493405 29820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:09.493628 29820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:09.495781 29820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "08aaf41b2c7749cd8b563462bea4372a"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "9a744ace549c7cf178938edf3257d662"
server_key_iv: "50144621e6eb2a547cf8d7bbad71e03d"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.496335 29820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "08aaf41b2c7749cd8b563462bea4372a"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "9a744ace549c7cf178938edf3257d662"
server_key_iv: "50144621e6eb2a547cf8d7bbad71e03d"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.500043 29820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 13:54:09.504297 29834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.505863 29820 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:54:09.506016 29820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "08aaf41b2c7749cd8b563462bea4372a"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "9a744ace549c7cf178938edf3257d662"
server_key_iv: "50144621e6eb2a547cf8d7bbad71e03d"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.506176 29820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:09.516793 29820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:09.517472 29820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:09.517663 29820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:09.524649 29820 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:40803
I20260926 13:54:09.524677 29886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:40803 every 8 connection(s)
I20260926 13:54:09.525580 29820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:09.528713 29887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:09.532886 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29820
I20260926 13:54:09.533001 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:09.533319 11187 external_mini_cluster.cc:1428] Setting key b05e60e47eb656db52b9a4f5187dfc48
I20260926 13:54:09.534307 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a: Bootstrap starting.
I20260926 13:54:09.536633 29887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:09.537478 29887 log.cc:824] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:09.539710 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a: No bootstrap required, opened a new log
I20260926 13:54:09.543314 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.534686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.10.236.254:40803)
0926 13:54:09.535373 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.535393 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.535428 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:54:09.536111 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.536128 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.536513 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.536849 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:09.537213 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.538414 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.539669 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.540286 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.540435 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.541403 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:09.541411 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:09.541435 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:09.541820 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:09.541875 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:09.541895 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:09.541986 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.542207 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.542538 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":133,"threads_started":1}
I20260926 13:54:09.543769 29887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08aaf41b2c7749cd8b563462bea4372a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40803 } }
I20260926 13:54:09.544018 29887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:09.544090 29887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08aaf41b2c7749cd8b563462bea4372a, State: Initialized, Role: FOLLOWER
I20260926 13:54:09.544543 29887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08aaf41b2c7749cd8b563462bea4372a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40803 } }
I20260926 13:54:09.544695 29887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:09.544778 29887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:09.544876 29887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:09.545784 29887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08aaf41b2c7749cd8b563462bea4372a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40803 } }
I20260926 13:54:09.546123 29887 leader_election.cc:302] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08aaf41b2c7749cd8b563462bea4372a; no voters:
I20260926 13:54:09.546416 29887 leader_election.cc:288] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:09.546566 29892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:09.546859 29892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [term 1 LEADER]: Becoming Leader. State: Replica: 08aaf41b2c7749cd8b563462bea4372a, State: Running, Role: LEADER
I20260926 13:54:09.547187 29892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08aaf41b2c7749cd8b563462bea4372a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40803 } }
I20260926 13:54:09.547694 29887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:09.549077 29894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08aaf41b2c7749cd8b563462bea4372a. Latest consensus state: current_term: 1 leader_uuid: "08aaf41b2c7749cd8b563462bea4372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08aaf41b2c7749cd8b563462bea4372a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40803 } } }
I20260926 13:54:09.549064 29893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08aaf41b2c7749cd8b563462bea4372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08aaf41b2c7749cd8b563462bea4372a" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40803 } } }
I20260926 13:54:09.549279 29893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:09.549603 29894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:09.550093 29906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:09.553198 29906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:09.559139 29906 catalog_manager.cc:1347] Generated new cluster ID: 5c92fac1b5274437862c9135caf66952
I20260926 13:54:09.559211 29906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:09.582423 29906 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:09.583351 29906 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:09.603502 29906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a: Generated new TSK 0
I20260926 13:54:09.604405 29906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:09.610700 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40803
--builtin_ntp_servers=127.10.236.212:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:54:09.723099 29911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:09.723359 29911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:09.723397 29911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:09.727439 29911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:09.727550 29911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:09.727667 29911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:54:09.732499 29911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:09.733605 29911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:09.734992 29911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:09.744760 29919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.744858 29911 server_base.cc:1034] running on GCE node
W20260926 13:54:09.744771 29917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:09.745077 29916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.745486 29911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:09.746094 29911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:09.747283 29911 hybrid_clock.cc:648] HybridClock initialized: now 1790430849747236 us; error 51 us; skew 500 ppm
I20260926 13:54:09.749220 29911 webserver.cc:460] Webserver started at http://127.10.236.193:38979/ using document root <none> and password file <none>
I20260926 13:54:09.749816 29911 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:09.749878 29911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:09.750039 29911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:09.752131 29911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "ccdd208660b5423ca3e13fd9214162ca"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "8c6f177ff11c5763e233fccd7a08a5e1"
server_key_iv: "bfd3c7f2233630f56a4008b1725f4e94"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.752637 29911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "ccdd208660b5423ca3e13fd9214162ca"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "8c6f177ff11c5763e233fccd7a08a5e1"
server_key_iv: "bfd3c7f2233630f56a4008b1725f4e94"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.756313 29911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:54:09.758733 29925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.760046 29911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:54:09.760174 29911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "ccdd208660b5423ca3e13fd9214162ca"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "8c6f177ff11c5763e233fccd7a08a5e1"
server_key_iv: "bfd3c7f2233630f56a4008b1725f4e94"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.760279 29911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:09.800073 29911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:09.800829 29911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:09.801051 29911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:09.801663 29911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:09.802634 29911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:09.802700 29911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.802774 29911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:09.802814 29911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.812081 29911 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:40541
I20260926 13:54:09.812104 30038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:40541 every 8 connection(s)
I20260926 13:54:09.813102 29911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:54:09.818384 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 29911
I20260926 13:54:09.818498 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:54:09.818800 11187 external_mini_cluster.cc:1428] Setting key a6453d55db367d49c819d6e750228fcb
I20260926 13:54:09.820860 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40803
--builtin_ntp_servers=127.10.236.212:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:54:09.822840 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.814915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:60217 (local address 127.10.236.254:40803)
0926 13:54:09.815084 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.815088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:09.816175 (+ 1087us) server_negotiation.cc:407] Connection header received
0926 13:54:09.817578 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:09.817582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:09.817640 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:09.817742 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:09.818710 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.819414 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.820856 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.821145 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.821240 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.822346 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:09.822350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:09.822353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:09.822366 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:09.822386 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:09.822399 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:09.822489 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:09.822643 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 13:54:09.822703 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 13:54:09.823794 30041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.815300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40803 (local address 127.10.236.193:60217)
0926 13:54:09.816004 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:09.816056 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:09.817361 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:09.817920 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:09.817928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:09.818018 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:09.818546 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.818578 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.819561 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.819565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.820732 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:09.820742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:09.821278 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:09.821282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:09.821456 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:09.822087 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:09.822088 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:09.822128 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:09.822193 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:09.822491 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:09.822503 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:09.822512 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 13:54:09.822643 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 13:54:09.822957 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":165,"threads_started":1}
I20260926 13:54:09.825004 30039 heartbeater.cc:347] Connected to a master server at 127.10.236.254:40803
I20260926 13:54:09.825244 30039 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:09.825750 30039 heartbeater.cc:511] Master 127.10.236.254:40803 requested a full tablet report, sending...
I20260926 13:54:09.827406 29851 ts_manager.cc:194] Registered new tserver with Master: ccdd208660b5423ca3e13fd9214162ca (127.10.236.193:40541)
I20260926 13:54:09.829247 29851 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.193:60217
W20260926 13:54:09.934638 30042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:09.934924 30042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:09.934963 30042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:09.939132 30042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:09.939198 30042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:09.939253 30042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:54:09.944236 30042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:09.945290 30042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:09.946688 30042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:09.956571 30048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:09.956548 30047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:09.956548 30050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:09.956696 30042 server_base.cc:1034] running on GCE node
I20260926 13:54:09.957281 30042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:09.957945 30042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:09.959136 30042 hybrid_clock.cc:648] HybridClock initialized: now 1790430849959107 us; error 47 us; skew 500 ppm
I20260926 13:54:09.960978 30042 webserver.cc:460] Webserver started at http://127.10.236.194:46363/ using document root <none> and password file <none>
I20260926 13:54:09.961570 30042 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:09.961658 30042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:09.961882 30042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:09.963964 30042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "fa68e8871b614289985cddf2c0b19e1f"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "9ee5a79ab41432f415cd6d7c3e84d002"
server_key_iv: "c5a9a91350b0b1517833b3b603231306"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.964591 30042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "fa68e8871b614289985cddf2c0b19e1f"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "9ee5a79ab41432f415cd6d7c3e84d002"
server_key_iv: "c5a9a91350b0b1517833b3b603231306"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.968413 30042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:54:09.970912 30056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.972301 30042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:54:09.972437 30042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "fa68e8871b614289985cddf2c0b19e1f"
format_stamp: "Formatted at 2026-09-26 13:54:09 on dist-test-slave-r34j"
server_key: "9ee5a79ab41432f415cd6d7c3e84d002"
server_key_iv: "c5a9a91350b0b1517833b3b603231306"
server_key_version: "encryptionkey@0"
I20260926 13:54:09.972551 30042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:09.984757 30042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:09.985445 30042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:09.985639 30042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:09.986213 30042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:09.987226 30042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:09.987294 30042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.987365 30042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:09.987406 30042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:09.996515 30042 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:46737
I20260926 13:54:09.996541 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:46737 every 8 connection(s)
I20260926 13:54:09.997506 30042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:54:10.007418 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30042
I20260926 13:54:10.007619 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:54:10.007913 11187 external_mini_cluster.cc:1428] Setting key b4cf8db09e3e18de3fe7475614aefa28
I20260926 13:54:10.008819 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:09.999555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:53189 (local address 127.10.236.254:40803)
0926 13:54:09.999732 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:09.999736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.000904 (+ 1168us) server_negotiation.cc:407] Connection header received
0926 13:54:10.002611 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.002616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.002694 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.002800 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:10.003945 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.004830 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.006556 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.006798 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.006907 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.008177 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:10.008181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:10.008185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:10.008201 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:10.008228 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:10.008240 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:10.008328 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.008567 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.008660 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 13:54:10.009915 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.000001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40803 (local address 127.10.236.194:53189)
0926 13:54:10.000730 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.000775 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.002318 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.003026 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.003036 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.003140 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:10.003795 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.003820 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.004991 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.004995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.006380 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.006390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.006986 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.006990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.007157 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.007884 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:10.007886 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:10.007930 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:10.008002 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:10.008349 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:10.008357 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:10.008384 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.008540 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.008843 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":116,"threads_started":1}
I20260926 13:54:10.010203 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40803
--builtin_ntp_servers=127.10.236.212:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:54:10.011327 30170 heartbeater.cc:347] Connected to a master server at 127.10.236.254:40803
I20260926 13:54:10.011632 30170 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:10.012395 30170 heartbeater.cc:511] Master 127.10.236.254:40803 requested a full tablet report, sending...
I20260926 13:54:10.013942 29851 ts_manager.cc:194] Registered new tserver with Master: fa68e8871b614289985cddf2c0b19e1f (127.10.236.194:46737)
I20260926 13:54:10.014571 29851 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.194:53189
W20260926 13:54:10.137074 30173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:10.137411 30173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:10.137492 30173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:10.142531 30173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:10.142622 30173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:10.142719 30173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:54:10.149001 30173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:10.150270 30173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:10.151757 30173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:10.161672 30181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:10.161692 30178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:10.161803 30173 server_base.cc:1034] running on GCE node
W20260926 13:54:10.162007 30179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:10.162439 30173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:10.163082 30173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:10.164321 30173 hybrid_clock.cc:648] HybridClock initialized: now 1790430850164297 us; error 34 us; skew 500 ppm
I20260926 13:54:10.166366 30173 webserver.cc:460] Webserver started at http://127.10.236.195:38695/ using document root <none> and password file <none>
I20260926 13:54:10.166980 30173 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:10.167076 30173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:10.167294 30173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:10.169445 30173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "66cc6013d8b447bdb0e8429f01cd32a9"
format_stamp: "Formatted at 2026-09-26 13:54:10 on dist-test-slave-r34j"
server_key: "342c2956952730bae9cfff7182eb4209"
server_key_iv: "32531b66c156a4448fdd119b147972b7"
server_key_version: "encryptionkey@0"
I20260926 13:54:10.170027 30173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "66cc6013d8b447bdb0e8429f01cd32a9"
format_stamp: "Formatted at 2026-09-26 13:54:10 on dist-test-slave-r34j"
server_key: "342c2956952730bae9cfff7182eb4209"
server_key_iv: "32531b66c156a4448fdd119b147972b7"
server_key_version: "encryptionkey@0"
I20260926 13:54:10.173785 30173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:54:10.176568 30187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:10.178038 30173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 13:54:10.178201 30173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "66cc6013d8b447bdb0e8429f01cd32a9"
format_stamp: "Formatted at 2026-09-26 13:54:10 on dist-test-slave-r34j"
server_key: "342c2956952730bae9cfff7182eb4209"
server_key_iv: "32531b66c156a4448fdd119b147972b7"
server_key_version: "encryptionkey@0"
I20260926 13:54:10.178323 30173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:10.195694 30173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:10.196440 30173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:10.196745 30173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:10.197446 30173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:10.198498 30173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:10.198657 30173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:10.198781 30173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:10.198838 30173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:10.208151 30173 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:41535
I20260926 13:54:10.208194 30300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:41535 every 8 connection(s)
I20260926 13:54:10.209158 30173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:54:10.218609 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30173
I20260926 13:54:10.218856 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:54:10.219322 11187 external_mini_cluster.cc:1428] Setting key 1e06037cbf0d1a90c3e5d55ba8c16823
I20260926 13:54:10.219354 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.211266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:39473 (local address 127.10.236.254:40803)
0926 13:54:10.211423 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.211427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.212618 (+ 1191us) server_negotiation.cc:407] Connection header received
0926 13:54:10.214057 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.214061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.214123 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.214231 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:10.215178 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.215785 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.216961 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.217195 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.217312 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.218721 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:10.218724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:10.218727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:10.218742 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:10.218763 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:10.218772 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:10.218861 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.219123 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.219194 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:54:10.220551 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.211625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40803 (local address 127.10.236.195:39473)
0926 13:54:10.212401 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.212481 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.213820 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.214414 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.214421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.214513 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:10.215021 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.215034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.215914 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.215917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.216828 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.216837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.217356 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.217360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.217542 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.218296 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:10.218298 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:10.218342 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:10.218413 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:10.218879 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:10.218892 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:10.218902 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.219071 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.219403 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":120,"threads_started":1}
I20260926 13:54:10.221840 30301 heartbeater.cc:347] Connected to a master server at 127.10.236.254:40803
I20260926 13:54:10.222117 30301 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:10.222654 30301 heartbeater.cc:511] Master 127.10.236.254:40803 requested a full tablet report, sending...
I20260926 13:54:10.223820 29850 ts_manager.cc:194] Registered new tserver with Master: 66cc6013d8b447bdb0e8429f01cd32a9 (127.10.236.195:41535)
I20260926 13:54:10.224422 29850 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.195:39473
I20260926 13:54:10.233698 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:54:10.239662 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.235829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.10.236.254:40803)
0926 13:54:10.235953 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.235956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.236110 (+ 154us) server_negotiation.cc:407] Connection header received
0926 13:54:10.236222 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.236224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.236264 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.236334 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:10.236632 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.237161 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.238068 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.238288 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.238368 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.239187 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:10.239190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:10.239192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:10.239204 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:10.239223 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:10.239230 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:10.239327 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.239439 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.239473 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:54:10.242892 29850 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:54:10.245977 29850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:54:10.261277 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.255068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.10.236.195:41535)
0926 13:54:10.255703 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.255718 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.255815 (+ 97us) server_negotiation.cc:407] Connection header received
0926 13:54:10.256086 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.256093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.256326 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.256510 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:10.257119 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.258305 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.259949 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.260490 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.260643 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.260837 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.260971 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.261075 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":162,"threads_started":1}
I20260926 13:54:10.261308 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.255368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40541 (local address 127.0.0.1:53996)
0926 13:54:10.255982 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.255997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.256091 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.256731 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.256734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.256755 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:10.256955 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.256963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.258554 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.258557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.259621 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.259634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.260892 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.260896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.260980 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.261002 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.261072 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.261137 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":147,"thread_start_us":465,"threads_started":1}
I20260926 13:54:10.261313 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.254854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:41535 (local address 127.0.0.1:42238)
0926 13:54:10.255660 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.255711 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.255867 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.256699 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.256705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.256733 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:10.256950 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.256962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.258447 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.258451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.259766 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.259777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.260640 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.260644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.260771 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.260813 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.260941 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.261076 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":35,"spinlock_wait_cycles":1920,"thread_start_us":86,"threads_started":1}
I20260926 13:54:10.261967 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.255322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.10.236.193:40541)
0926 13:54:10.255934 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.255948 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.256082 (+ 134us) server_negotiation.cc:407] Connection header received
0926 13:54:10.256233 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.256240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.256435 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.256579 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:10.257097 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.258436 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.259759 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.260545 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.260658 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.260836 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.261693 (+ 857us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.261796 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":151,"threads_started":1}
I20260926 13:54:10.262398 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.256117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:46737 (local address 127.0.0.1:46232)
0926 13:54:10.256884 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.256899 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.256991 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.257430 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.257433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.257453 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:10.257565 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.257572 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.258657 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.258660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.260686 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.260695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.261994 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.261998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.262115 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.262137 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.262216 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.262282 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260926 13:54:10.262545 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.256326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46232 (local address 127.10.236.194:46737)
0926 13:54:10.256744 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.256755 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.256986 (+ 231us) server_negotiation.cc:407] Connection header received
0926 13:54:10.257108 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.257116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.257276 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.257440 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:10.257696 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.258517 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.260822 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.261627 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.261722 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.261891 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.262254 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.262376 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":112,"threads_started":1}
I20260926 13:54:10.264818 30235 tablet_service.cc:1467] Processing CreateTablet for tablet e5bcfe6b054a4459a200fa3c946d9637 (DEFAULT_TABLE table=test-table [id=efaa2909cf804f62ba75949de1b3be11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:10.264883 29973 tablet_service.cc:1467] Processing CreateTablet for tablet e5bcfe6b054a4459a200fa3c946d9637 (DEFAULT_TABLE table=test-table [id=efaa2909cf804f62ba75949de1b3be11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:10.264986 30104 tablet_service.cc:1467] Processing CreateTablet for tablet e5bcfe6b054a4459a200fa3c946d9637 (DEFAULT_TABLE table=test-table [id=efaa2909cf804f62ba75949de1b3be11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:10.265923 30235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bcfe6b054a4459a200fa3c946d9637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:10.265936 29973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bcfe6b054a4459a200fa3c946d9637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:10.265936 30104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bcfe6b054a4459a200fa3c946d9637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:10.272909 30318 tablet_bootstrap.cc:492] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Bootstrap starting.
I20260926 13:54:10.273706 30320 tablet_bootstrap.cc:492] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: Bootstrap starting.
I20260926 13:54:10.273857 30319 tablet_bootstrap.cc:492] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: Bootstrap starting.
I20260926 13:54:10.275146 30318 tablet_bootstrap.cc:654] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:10.275840 30320 tablet_bootstrap.cc:654] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:10.275840 30319 tablet_bootstrap.cc:654] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:10.276187 30318 log.cc:824] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:10.276659 30319 log.cc:824] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:10.276659 30320 log.cc:824] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:10.278499 30318 tablet_bootstrap.cc:492] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: No bootstrap required, opened a new log
I20260926 13:54:10.278499 30319 tablet_bootstrap.cc:492] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: No bootstrap required, opened a new log
I20260926 13:54:10.278702 30319 ts_tablet_manager.cc:1397] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 13:54:10.278702 30318 ts_tablet_manager.cc:1397] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:54:10.280535 30320 tablet_bootstrap.cc:492] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: No bootstrap required, opened a new log
I20260926 13:54:10.280738 30320 ts_tablet_manager.cc:1397] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 13:54:10.283336 30318 raft_consensus.cc:348] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.283336 30319 raft_consensus.cc:348] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.283658 30318 raft_consensus.cc:374] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:10.283658 30319 raft_consensus.cc:374] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:10.283723 30319 raft_consensus.cc:729] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa68e8871b614289985cddf2c0b19e1f, State: Initialized, Role: FOLLOWER
I20260926 13:54:10.283751 30318 raft_consensus.cc:729] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66cc6013d8b447bdb0e8429f01cd32a9, State: Initialized, Role: FOLLOWER
I20260926 13:54:10.284256 30318 consensus_queue.cc:260] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.284256 30319 consensus_queue.cc:260] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.284567 30320 raft_consensus.cc:348] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.284866 30320 raft_consensus.cc:374] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:10.284972 30320 raft_consensus.cc:729] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdd208660b5423ca3e13fd9214162ca, State: Initialized, Role: FOLLOWER
I20260926 13:54:10.285491 30319 ts_tablet_manager.cc:1428] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 13:54:10.285491 30318 ts_tablet_manager.cc:1428] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260926 13:54:10.285467 30320 consensus_queue.cc:260] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.285892 30170 heartbeater.cc:503] Master 127.10.236.254:40803 was elected leader, sending a full tablet report...
I20260926 13:54:10.285976 30301 heartbeater.cc:503] Master 127.10.236.254:40803 was elected leader, sending a full tablet report...
I20260926 13:54:10.286439 30320 ts_tablet_manager.cc:1428] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 13:54:10.286815 30039 heartbeater.cc:503] Master 127.10.236.254:40803 was elected leader, sending a full tablet report...
W20260926 13:54:10.314298 30040 tablet.cc:2369] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:54:10.389855 30325 raft_consensus.cc:482] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:10.390161 30325 raft_consensus.cc:504] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.391425 30325 leader_election.cc:288] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66cc6013d8b447bdb0e8429f01cd32a9 (127.10.236.195:41535), ccdd208660b5423ca3e13fd9214162ca (127.10.236.193:40541)
I20260926 13:54:10.395851 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.391841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:46723 (local address 127.10.236.195:41535)
0926 13:54:10.391982 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.391988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.392005 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:54:10.392261 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.392265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.392318 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.392416 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:10.392821 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.393523 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.394677 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.395199 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.395302 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.395344 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.395629 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.395704 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:54:10.395850 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.391758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:41535 (local address 127.10.236.194:46723)
0926 13:54:10.391907 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.391925 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.392041 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.392498 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.392501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.392527 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:10.392665 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.392673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.393658 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.393661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.394535 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.394545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.395343 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.395347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.395467 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.395529 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.395611 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.395692 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":1}
I20260926 13:54:10.396092 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.391774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40541 (local address 127.10.236.194:37561)
0926 13:54:10.392385 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.392400 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:10.392497 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:10.392820 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:10.392823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:10.392844 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:10.392978 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.392985 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.393797 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.393800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.394729 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:10.394739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.395633 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.395638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:10.395741 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.395766 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 13:54:10.395853 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 13:54:10.395931 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":124,"thread_start_us":157,"threads_started":1}
I20260926 13:54:10.396129 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.391900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:37561 (local address 127.10.236.193:40541)
0926 13:54:10.392030 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.392037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.392504 (+ 467us) server_negotiation.cc:407] Connection header received
0926 13:54:10.392599 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.392602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.392659 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.392762 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:10.393126 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.393676 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.394883 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.395459 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.395628 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.395673 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.395891 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.395984 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:54:10.396643 30255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bcfe6b054a4459a200fa3c946d9637" candidate_uuid: "fa68e8871b614289985cddf2c0b19e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" is_pre_election: true
I20260926 13:54:10.396857 29993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bcfe6b054a4459a200fa3c946d9637" candidate_uuid: "fa68e8871b614289985cddf2c0b19e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccdd208660b5423ca3e13fd9214162ca" is_pre_election: true
I20260926 13:54:10.396983 30255 raft_consensus.cc:2454] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa68e8871b614289985cddf2c0b19e1f in term 0.
I20260926 13:54:10.397138 29993 raft_consensus.cc:2454] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa68e8871b614289985cddf2c0b19e1f in term 0.
I20260926 13:54:10.397542 30059 leader_election.cc:302] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66cc6013d8b447bdb0e8429f01cd32a9, fa68e8871b614289985cddf2c0b19e1f; no voters:
I20260926 13:54:10.397842 30325 raft_consensus.cc:2781] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:54:10.397933 30325 raft_consensus.cc:482] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:54:10.397979 30325 raft_consensus.cc:3037] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:10.399056 30325 raft_consensus.cc:504] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.399428 30325 leader_election.cc:288] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [CANDIDATE]: Term 1 election: Requested vote from peers 66cc6013d8b447bdb0e8429f01cd32a9 (127.10.236.195:41535), ccdd208660b5423ca3e13fd9214162ca (127.10.236.193:40541)
I20260926 13:54:10.399788 30255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bcfe6b054a4459a200fa3c946d9637" candidate_uuid: "fa68e8871b614289985cddf2c0b19e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66cc6013d8b447bdb0e8429f01cd32a9"
I20260926 13:54:10.399924 30255 raft_consensus.cc:3037] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:10.399932 29993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bcfe6b054a4459a200fa3c946d9637" candidate_uuid: "fa68e8871b614289985cddf2c0b19e1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccdd208660b5423ca3e13fd9214162ca"
I20260926 13:54:10.400082 29993 raft_consensus.cc:3037] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:10.401073 30255 raft_consensus.cc:2454] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa68e8871b614289985cddf2c0b19e1f in term 1.
I20260926 13:54:10.401214 29993 raft_consensus.cc:2454] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa68e8871b614289985cddf2c0b19e1f in term 1.
I20260926 13:54:10.401455 30059 leader_election.cc:302] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66cc6013d8b447bdb0e8429f01cd32a9, fa68e8871b614289985cddf2c0b19e1f; no voters:
I20260926 13:54:10.401623 30325 raft_consensus.cc:2781] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:10.401932 30325 raft_consensus.cc:686] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [term 1 LEADER]: Becoming Leader. State: Replica: fa68e8871b614289985cddf2c0b19e1f, State: Running, Role: LEADER
I20260926 13:54:10.402285 30325 consensus_queue.cc:237] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } }
I20260926 13:54:10.405809 29850 catalog_manager.cc:5495] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f reported cstate change: term changed from 0 to 1, leader changed from <none> to fa68e8871b614289985cddf2c0b19e1f (127.10.236.194). New cstate: current_term: 1 leader_uuid: "fa68e8871b614289985cddf2c0b19e1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa68e8871b614289985cddf2c0b19e1f" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46737 } health_report { overall_health: HEALTHY } } }
I20260926 13:54:10.453873 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.450242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46234 (local address 127.10.236.194:46737)
0926 13:54:10.450360 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:10.450363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:10.450377 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:54:10.450477 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:10.450480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:10.450541 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:10.450633 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:10.450871 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.451412 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.452433 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:10.452651 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:10.452736 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:10.453038 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:10.453440 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:10.453558 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:10.453710 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 13:54:10.453743 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:54:10.460071 29993 raft_consensus.cc:1261] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 1 FOLLOWER]: Refusing update from remote peer fa68e8871b614289985cddf2c0b19e1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:10.460071 30255 raft_consensus.cc:1261] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 1 FOLLOWER]: Refusing update from remote peer fa68e8871b614289985cddf2c0b19e1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260926 13:54:10.460362 30302 tablet.cc:2369] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:54:10.460834 30325 consensus_queue.cc:1035] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "66cc6013d8b447bdb0e8429f01cd32a9" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:10.461053 30328 consensus_queue.cc:1035] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccdd208660b5423ca3e13fd9214162ca" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:10.468421 30332 mvcc.cc:205] Tried to move back new op lower bound from 7333604763476561920 to 7333604763250769920. Current Snapshot: MvccSnapshot[applied={T|T < 7333604763476561920}]
I20260926 13:54:10.470536 30333 mvcc.cc:205] Tried to move back new op lower bound from 7333604763476561920 to 7333604763250769920. Current Snapshot: MvccSnapshot[applied={T|T < 7333604763476561920}]
I20260926 13:54:10.478089 29850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:10.478289 29850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:10.480885 29850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08aaf41b2c7749cd8b563462bea4372a: Sending DeleteTablet for 3 replicas of tablet e5bcfe6b054a4459a200fa3c946d9637
I20260926 13:54:10.481740 30235 tablet_service.cc:1514] Processing DeleteTablet for tablet e5bcfe6b054a4459a200fa3c946d9637 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:10 UTC) from {username='slave'} at 127.0.0.1:42238
I20260926 13:54:10.481945 30104 tablet_service.cc:1514] Processing DeleteTablet for tablet e5bcfe6b054a4459a200fa3c946d9637 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:10 UTC) from {username='slave'} at 127.0.0.1:46232
I20260926 13:54:10.482299 30340 tablet_replica.cc:330] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: stopping tablet replica
I20260926 13:54:10.482419 29973 tablet_service.cc:1514] Processing DeleteTablet for tablet e5bcfe6b054a4459a200fa3c946d9637 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:10 UTC) from {username='slave'} at 127.0.0.1:53996
I20260926 13:54:10.482674 30340 raft_consensus.cc:2229] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:10.482931 30342 tablet_replica.cc:330] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca: stopping tablet replica
I20260926 13:54:10.483035 30340 raft_consensus.cc:2258] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:10.483346 30342 raft_consensus.cc:2229] T e5bcfe6b054a4459a200fa3c946d9637 P ccdd208660b5423ca3e13fd9214162ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:10.483563 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29911
I20260926 13:54:10.485481 30340 ts_tablet_manager.cc:1905] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:10.488171 30340 ts_tablet_manager.cc:1918] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:10.488292 30340 log.cc:1196] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/e5bcfe6b054a4459a200fa3c946d9637
I20260926 13:54:10.488586 30340 ts_tablet_manager.cc:1939] T e5bcfe6b054a4459a200fa3c946d9637 P 66cc6013d8b447bdb0e8429f01cd32a9: Deleting consensus metadata
I20260926 13:54:10.489799 29837 catalog_manager.cc:4841] TS 66cc6013d8b447bdb0e8429f01cd32a9 (127.10.236.195:41535): tablet e5bcfe6b054a4459a200fa3c946d9637 (table test-table [id=efaa2909cf804f62ba75949de1b3be11]) successfully deleted
I20260926 13:54:10.491797 30341 tablet_replica.cc:330] T e5bcfe6b054a4459a200fa3c946d9637 P fa68e8871b614289985cddf2c0b19e1f: stopping tablet replica
I20260926 13:54:10.492010 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30042
W20260926 13:54:10.492134 29838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:40541 (error 108)
I20260926 13:54:10.493551 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.493264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:40541 (local address 127.0.0.1:54012)
0926 13:54:10.493410 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.493478 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:40541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 13:54:10.493786 29838 catalog_manager.cc:4584] TS ccdd208660b5423ca3e13fd9214162ca (127.10.236.193:40541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5bcfe6b054a4459a200fa3c946d9637: Network error: Client connection negotiation failed: client connection to 127.10.236.193:40541: connect: Connection refused (error 111)
I20260926 13:54:10.498251 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30173
I20260926 13:54:10.499315 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:10.498317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:46737 (local address 127.0.0.1:46242)
0926 13:54:10.499148 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:10.499214 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.194:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":729}
I20260926 13:54:10.504567 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 29820
2026-09-26T13:54:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1177 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:10 dist-test-slave-r34j krb5kdc[30346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:10 dist-test-slave-r34j krb5kdc[30346](info): set up 2 sockets
Sep 26 13:54:10 dist-test-slave-r34j krb5kdc[30346](info): commencing operation
krb5kdc: starting...
W20260926 13:54:11.563820 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:11 dist-test-slave-r34j krb5kdc[30346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:11.694538 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:40049
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:40049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:11.804869 30359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:11.805202 30359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:11.805282 30359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:11.809441 30359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:11.809554 30359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:11.809593 30359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:11.809631 30359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:11.809669 30359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:11.815083 30359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:40049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:40049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:11.816239 30359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:11.817584 30359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:11.826601 30367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:11.826666 30365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:11.826726 30359 server_base.cc:1034] running on GCE node
W20260926 13:54:11.826689 30364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:11.827435 30359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:11.828611 30359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:11.829813 30359 hybrid_clock.cc:648] HybridClock initialized: now 1790430851829785 us; error 40 us; skew 500 ppm
Sep 26 13:54:11 dist-test-slave-r34j krb5kdc[30346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:11.833039 30359 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:11.834004 30359 webserver.cc:460] Webserver started at http://127.10.236.254:32979/ using document root <none> and password file <none>
I20260926 13:54:11.834609 30359 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:11.834699 30359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:11.834913 30359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:11.836999 30359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "4048481908754fdb975e4b5532b28150"
format_stamp: "Formatted at 2026-09-26 13:54:11 on dist-test-slave-r34j"
server_key: "83f50468008e3240b8819dd0110a552d"
server_key_iv: "4cb475b3c898a2311408df8e590d76bb"
server_key_version: "encryptionkey@0"
I20260926 13:54:11.837553 30359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "4048481908754fdb975e4b5532b28150"
format_stamp: "Formatted at 2026-09-26 13:54:11 on dist-test-slave-r34j"
server_key: "83f50468008e3240b8819dd0110a552d"
server_key_iv: "4cb475b3c898a2311408df8e590d76bb"
server_key_version: "encryptionkey@0"
I20260926 13:54:11.841287 30359 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:54:11.843796 30374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:11.845146 30359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:54:11.845285 30359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "4048481908754fdb975e4b5532b28150"
format_stamp: "Formatted at 2026-09-26 13:54:11 on dist-test-slave-r34j"
server_key: "83f50468008e3240b8819dd0110a552d"
server_key_iv: "4cb475b3c898a2311408df8e590d76bb"
server_key_version: "encryptionkey@0"
I20260926 13:54:11.845398 30359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:11.860255 30359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:11.863225 30359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:11.863456 30359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:11.870711 30359 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:40049
I20260926 13:54:11.870766 30426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:40049 every 8 connection(s)
I20260926 13:54:11.871819 30359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:11.874643 30427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:11.880177 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150: Bootstrap starting.
I20260926 13:54:11.881487 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30359
I20260926 13:54:11.881666 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:11.881938 11187 external_mini_cluster.cc:1428] Setting key a9df2e422aa4186a92abb7fa3b207f07
I20260926 13:54:11.882921 30427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:11.883744 30427 log.cc:824] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:11.885938 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150: No bootstrap required, opened a new log
I20260926 13:54:11.890391 30427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4048481908754fdb975e4b5532b28150" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40049 } }
I20260926 13:54:11.890702 30427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:11.890796 30427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4048481908754fdb975e4b5532b28150, State: Initialized, Role: FOLLOWER
Sep 26 13:54:11 dist-test-slave-r34j krb5kdc[30346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:11.891237 30427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4048481908754fdb975e4b5532b28150" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40049 } }
I20260926 13:54:11.891404 30427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:11.891469 30427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:11.891613 30427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:11.892522 30427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4048481908754fdb975e4b5532b28150" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40049 } }
I20260926 13:54:11.892880 30427 leader_election.cc:302] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4048481908754fdb975e4b5532b28150; no voters:
I20260926 13:54:11.893225 30427 leader_election.cc:288] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:11.893288 30432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:11.893487 30432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [term 1 LEADER]: Becoming Leader. State: Replica: 4048481908754fdb975e4b5532b28150, State: Running, Role: LEADER
I20260926 13:54:11.893793 30432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4048481908754fdb975e4b5532b28150" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40049 } }
I20260926 13:54:11.894601 30427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:11.895710 30434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4048481908754fdb975e4b5532b28150. Latest consensus state: current_term: 1 leader_uuid: "4048481908754fdb975e4b5532b28150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4048481908754fdb975e4b5532b28150" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40049 } } }
I20260926 13:54:11.896109 30434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:11.895992 30433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4048481908754fdb975e4b5532b28150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4048481908754fdb975e4b5532b28150" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40049 } } }
I20260926 13:54:11.896903 30433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:11.897176 30443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:11.898178 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:11.883077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.10.236.254:40049)
0926 13:54:11.883547 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:11.883562 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:11.883597 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:54:11.884293 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:11.884311 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:11.884626 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:11.884902 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:11.885797 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:11.887058 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:11.888285 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:11.888763 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:11.888880 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:11.892007 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:11.892012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:11.892026 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:11.892058 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:11.894130 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:11.894581 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:11.894589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:11.894595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:11.894765 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:11.895125 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:11.895128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:11.895130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:11.895733 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:11.895948 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:11.896877 (+ 929us) server_negotiation.cc:299] Negotiation successful
0926 13:54:11.897218 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":133,"threads_started":1}
I20260926 13:54:11.900422 30443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:11.906333 30443 catalog_manager.cc:1347] Generated new cluster ID: 4a9d96158620449fb57e9b9beb7d5362
I20260926 13:54:11.906438 30443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:11.924898 30443 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:11.926085 30443 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:11.931097 30443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150: Generated new TSK 0
I20260926 13:54:11.931875 30443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.236.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
Entry for principal kudu/127.10.236.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab.
I20260926 13:54:11.967332 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40049
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:12.083590 30453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:12.083849 30453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:12.083894 30453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:12.088050 30453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:12.088119 30453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:12.088178 30453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:54:12.093489 30453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.193.keytab
--principal=kudu/127.10.236.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:12.094530 30453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:12.095566 30453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:12.105199 30458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:12.105187 30459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:12.105298 30453 server_base.cc:1034] running on GCE node
W20260926 13:54:12.105525 30461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:12.105949 30453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:12.106586 30453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:12.107756 30453 hybrid_clock.cc:648] HybridClock initialized: now 1790430852107741 us; error 29 us; skew 500 ppm
Sep 26 13:54:12 dist-test-slave-r34j krb5kdc[30346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430852, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:12.110505 30453 init.cc:377] Logged in from keytab as kudu/127.10.236.193@KRBTEST.COM (short username kudu)
I20260926 13:54:12.111363 30453 webserver.cc:460] Webserver started at http://127.10.236.193:36191/ using document root <none> and password file <none>
I20260926 13:54:12.112003 30453 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:12.112141 30453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:12.112370 30453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:12.114439 30453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "dc1076f168e445e9898c773187241cbb"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "e906ec84d2bf82d9c8f6efe12a656922"
server_key_iv: "0a8072e751100fd036a4a4cad6409c3e"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.115029 30453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "dc1076f168e445e9898c773187241cbb"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "e906ec84d2bf82d9c8f6efe12a656922"
server_key_iv: "0a8072e751100fd036a4a4cad6409c3e"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.118801 30453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 13:54:12.121284 30468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.122505 30453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:12.122601 30453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "dc1076f168e445e9898c773187241cbb"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "e906ec84d2bf82d9c8f6efe12a656922"
server_key_iv: "0a8072e751100fd036a4a4cad6409c3e"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.122741 30453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:12.137024 30453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:12.140554 30453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:12.140764 30453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:12.141367 30453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:12.142220 30453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:12.142266 30453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.142308 30453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:12.142362 30453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.152381 30453 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:33095
I20260926 13:54:12.152475 30581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:33095 every 8 connection(s)
I20260926 13:54:12.153374 30453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:54:12.154284 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30453
I20260926 13:54:12.154433 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:54:12.154745 11187 external_mini_cluster.cc:1428] Setting key c32cc6aef895a8f3e2dcc5cb004f4308
WARNING: no policy specified for kudu/127.10.236.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.194@KRBTEST.COM" created.
Sep 26 13:54:12 dist-test-slave-r34j krb5kdc[30346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430852, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.193@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:12.172674 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.155654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:56961 (local address 127.10.236.254:40049)
0926 13:54:12.155785 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.155788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.156897 (+ 1109us) server_negotiation.cc:407] Connection header received
0926 13:54:12.158847 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.158852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.158923 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.159099 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:12.160454 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.161395 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.162682 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.163087 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.163210 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.168740 (+ 5530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:12.168744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:12.168746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:12.168784 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:12.170373 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.170951 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.170954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.170956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.171080 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.171437 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.171441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.171443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.171701 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:12.171811 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.172336 (+ 525us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.172490 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:54:12.173586 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.156096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40049 (local address 127.10.236.193:56961)
0926 13:54:12.156718 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.156775 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.158587 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.159292 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.159301 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.159779 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:12.160282 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.160296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.161549 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.161552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.162543 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.162553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.163248 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.163251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.163399 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.164092 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:12.164109 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:12.168538 (+ 4429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:12.170509 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:12.170514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:12.170527 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:12.170822 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:12.171180 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:12.171186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:12.171189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:12.171332 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:12.171808 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:12.171815 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:12.172086 (+ 271us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.172275 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.172637 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":158,"threads_started":1}
I20260926 13:54:12.174943 30582 heartbeater.cc:347] Connected to a master server at 127.10.236.254:40049
I20260926 13:54:12.175216 30582 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:12.175777 30582 heartbeater.cc:511] Master 127.10.236.254:40049 requested a full tablet report, sending...
I20260926 13:54:12.177402 30391 ts_manager.cc:194] Registered new tserver with Master: dc1076f168e445e9898c773187241cbb (127.10.236.193:33095)
I20260926 13:54:12.178705 30391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.193@KRBTEST.COM'} at 127.10.236.193:56961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
Entry for principal kudu/127.10.236.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab.
I20260926 13:54:12.186381 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40049
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:12.299243 30587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:12.299582 30587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:12.299633 30587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:12.303697 30587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:12.303747 30587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:12.303804 30587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:54:12.309022 30587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.194.keytab
--principal=kudu/127.10.236.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:12.310006 30587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:12.310947 30587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:12.320708 30593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:12.320715 30595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:12.320752 30592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:12.320789 30587 server_base.cc:1034] running on GCE node
I20260926 13:54:12.321403 30587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:12.322122 30587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:12.323328 30587 hybrid_clock.cc:648] HybridClock initialized: now 1790430852323320 us; error 37 us; skew 500 ppm
Sep 26 13:54:12 dist-test-slave-r34j krb5kdc[30346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430852, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:12.326198 30587 init.cc:377] Logged in from keytab as kudu/127.10.236.194@KRBTEST.COM (short username kudu)
I20260926 13:54:12.327176 30587 webserver.cc:460] Webserver started at http://127.10.236.194:45553/ using document root <none> and password file <none>
I20260926 13:54:12.327770 30587 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:12.327831 30587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:12.327998 30587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:12.329968 30587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "6720307717f4482ca7f05539ed6325b8"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "4569c2d866143b6bb8254295e88c378e"
server_key_iv: "1c2cbb9e06ce5fef59b778271bf8cad6"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.330472 30587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "6720307717f4482ca7f05539ed6325b8"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "4569c2d866143b6bb8254295e88c378e"
server_key_iv: "1c2cbb9e06ce5fef59b778271bf8cad6"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.334184 30587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 13:54:12.336629 30602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.337824 30587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:54:12.337937 30587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "6720307717f4482ca7f05539ed6325b8"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "4569c2d866143b6bb8254295e88c378e"
server_key_iv: "1c2cbb9e06ce5fef59b778271bf8cad6"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.338023 30587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:12.356123 30587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:12.359309 30587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:12.359457 30587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:12.360105 30587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:12.360872 30587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:12.360920 30587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.360958 30587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:12.360973 30587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.369828 30587 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:36539
I20260926 13:54:12.369853 30715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:36539 every 8 connection(s)
I20260926 13:54:12.370743 30587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:54:12.373163 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30587
I20260926 13:54:12.373306 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:54:12.373641 11187 external_mini_cluster.cc:1428] Setting key 6f43e8f24c3e1141920f68bfc2a61da4
Sep 26 13:54:12 dist-test-slave-r34j krb5kdc[30346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430852, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.194@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.195@KRBTEST.COM" created.
I20260926 13:54:12.386781 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.372624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:33035 (local address 127.10.236.254:40049)
0926 13:54:12.372793 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.372798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.373793 (+ 995us) server_negotiation.cc:407] Connection header received
0926 13:54:12.375371 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.375374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.375433 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.375572 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:12.376737 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.377444 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.379322 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.379670 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.379761 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.382726 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:12.382733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:12.382735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:12.382771 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:12.384458 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.385065 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.385073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.385074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.385151 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.385493 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.385497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.385498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.385696 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:12.385774 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.386383 (+ 609us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.386565 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 13:54:12.387713 30718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.373068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40049 (local address 127.10.236.194:33035)
0926 13:54:12.373624 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.373666 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.375144 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.375721 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.375728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.376084 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:12.376577 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.376591 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.377606 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.377612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.379128 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.379143 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.379836 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.379840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.380056 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.380709 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:12.380735 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:12.382542 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:12.384603 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:12.384609 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:12.384625 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:12.384944 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:12.385244 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:12.385246 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:12.385248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:12.385381 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:12.385800 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:12.385805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:12.386153 (+ 348us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.386318 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.386681 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":154,"threads_started":1}
I20260926 13:54:12.389150 30716 heartbeater.cc:347] Connected to a master server at 127.10.236.254:40049
I20260926 13:54:12.389427 30716 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:12.389941 30716 heartbeater.cc:511] Master 127.10.236.254:40049 requested a full tablet report, sending...
I20260926 13:54:12.391117 30391 ts_manager.cc:194] Registered new tserver with Master: 6720307717f4482ca7f05539ed6325b8 (127.10.236.194:36539)
I20260926 13:54:12.391880 30391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.194@KRBTEST.COM'} at 127.10.236.194:33035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
Entry for principal kudu/127.10.236.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab.
I20260926 13:54:12.407714 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40049
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:12.519088 30721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:12.519398 30721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:12.519472 30721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:12.523834 30721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:12.523924 30721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:12.524013 30721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:54:12.529523 30721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.195.keytab
--principal=kudu/127.10.236.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:40049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:12.530607 30721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:12.531710 30721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:12.541666 30729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:12.541626 30726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:12.541625 30727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:12.541936 30721 server_base.cc:1034] running on GCE node
I20260926 13:54:12.542420 30721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:12.543051 30721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:12.544238 30721 hybrid_clock.cc:648] HybridClock initialized: now 1790430852544204 us; error 44 us; skew 500 ppm
Sep 26 13:54:12 dist-test-slave-r34j krb5kdc[30346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430852, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:12.547053 30721 init.cc:377] Logged in from keytab as kudu/127.10.236.195@KRBTEST.COM (short username kudu)
I20260926 13:54:12.547911 30721 webserver.cc:460] Webserver started at http://127.10.236.195:42281/ using document root <none> and password file <none>
I20260926 13:54:12.548518 30721 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:12.548579 30721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:12.548779 30721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:12.550788 30721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "cc174b40af93439fbb921c973499e259"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "2b187bc4d7d5bc23defdb6224f3cabd8"
server_key_iv: "89ec82fb399bc2ac957147c3b22060be"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.551296 30721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "cc174b40af93439fbb921c973499e259"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "2b187bc4d7d5bc23defdb6224f3cabd8"
server_key_iv: "89ec82fb399bc2ac957147c3b22060be"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.555050 30721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:54:12.557770 30736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.559010 30721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:12.559115 30721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "cc174b40af93439fbb921c973499e259"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "2b187bc4d7d5bc23defdb6224f3cabd8"
server_key_iv: "89ec82fb399bc2ac957147c3b22060be"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.559202 30721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:12.572340 30721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:12.575603 30721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:12.575796 30721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:12.576345 30721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:12.577101 30721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:12.577147 30721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.577185 30721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:12.577200 30721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.587980 30721 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:35315
I20260926 13:54:12.588027 30849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:35315 every 8 connection(s)
I20260926 13:54:12.588956 30721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:54:12.594549 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30721
I20260926 13:54:12.594705 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:54:12.595021 11187 external_mini_cluster.cc:1428] Setting key 013251eefdff9609f4d79c08651681f2
Sep 26 13:54:12 dist-test-slave-r34j krb5kdc[30346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430852, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.195@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:12.603574 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.590950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:41435 (local address 127.10.236.254:40049)
0926 13:54:12.591116 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.591121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.592028 (+ 907us) server_negotiation.cc:407] Connection header received
0926 13:54:12.593276 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.593279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.593357 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.593502 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:12.595178 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.596023 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.597458 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.597672 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.597754 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.600265 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:12.600268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:12.600270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:12.600294 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:12.601698 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.602199 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.602201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.602203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.602279 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.602568 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.602571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.602572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.602746 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:12.602818 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.603294 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.603390 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 13:54:12.604463 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.591328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40049 (local address 127.10.236.195:41435)
0926 13:54:12.591831 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.591873 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.593057 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.593716 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.593730 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.594316 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:12.594970 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.594986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.596192 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.596194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.597355 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.597364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.597799 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.597801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.597938 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.598528 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:12.598545 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:12.600103 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:12.601811 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:12.601815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:12.601828 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:12.602106 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:12.602359 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:12.602361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:12.602362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:12.602489 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:12.602833 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:12.602838 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:12.603076 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.603236 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.603604 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":178,"threads_started":1}
I20260926 13:54:12.605721 30850 heartbeater.cc:347] Connected to a master server at 127.10.236.254:40049
I20260926 13:54:12.605959 30850 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:12.606447 30850 heartbeater.cc:511] Master 127.10.236.254:40049 requested a full tablet report, sending...
I20260926 13:54:12.607589 30391 ts_manager.cc:194] Registered new tserver with Master: cc174b40af93439fbb921c973499e259 (127.10.236.195:35315)
I20260926 13:54:12.608263 30391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.236.195@KRBTEST.COM'} at 127.10.236.195:41435
I20260926 13:54:12.609138 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:54:12.618577 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.611177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.10.236.254:40049)
0926 13:54:12.611321 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.611326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.611524 (+ 198us) server_negotiation.cc:407] Connection header received
0926 13:54:12.611656 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.611660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.611701 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.611789 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:12.612484 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.612993 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.613850 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.614078 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.614179 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.615595 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:12.615599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:12.615601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:12.615622 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:12.616960 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.617365 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.617371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.617375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.617453 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:12.617720 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:12.617724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:12.617726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:12.617929 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:12.618067 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.618301 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.618411 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:54:12.621716 30391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:54:12.624791 30391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:54:12.639698 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.633621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:33095 (local address 127.0.0.1:44190)
0926 13:54:12.634605 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.634619 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.634771 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.635475 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.635477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.635490 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:12.635613 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.635620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.636958 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.636961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.638244 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.638262 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.639089 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.639093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.639213 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.639254 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.639396 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.639546 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"mutex_wait_us":4,"thread_start_us":129,"threads_started":1}
I20260926 13:54:12.639705 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.633717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44190 (local address 127.10.236.193:33095)
0926 13:54:12.634463 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.634476 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.634698 (+ 222us) server_negotiation.cc:407] Connection header received
0926 13:54:12.634926 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.634934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.635141 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.635271 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:12.635756 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.636818 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.638368 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.638956 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.639091 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.639232 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.639432 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.639562 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":87,"threads_started":1}
I20260926 13:54:12.641562 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.633605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:35315 (local address 127.0.0.1:42678)
0926 13:54:12.634192 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.634248 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.634397 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.635393 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.635399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.635425 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:12.635616 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.635621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.638245 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.638249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.639458 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.639469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.641153 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.641157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.641253 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.641276 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.641347 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.641416 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":45,"thread_start_us":76,"threads_started":1}
I20260926 13:54:12.641897 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.634786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.10.236.194:36539)
0926 13:54:12.635242 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.635252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.635756 (+ 504us) server_negotiation.cc:407] Connection header received
0926 13:54:12.635875 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.635879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.636004 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.636109 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:12.636406 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.637285 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.639766 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.640686 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.640817 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.640962 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.641664 (+ 702us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.641769 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":131,"threads_started":1}
I20260926 13:54:12.641587 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.634621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36539 (local address 127.0.0.1:55380)
0926 13:54:12.635596 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.635621 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.635756 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.636140 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.636143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.636165 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:12.636284 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.636292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.637475 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.637479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.639610 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.639624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.641045 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.641048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.641178 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.641207 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.641296 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.641389 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260926 13:54:12.642076 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.634069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.10.236.195:35315)
0926 13:54:12.634765 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.634784 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.634811 (+ 27us) server_negotiation.cc:407] Connection header received
0926 13:54:12.634920 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.634927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.635217 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.635397 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:12.636996 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.638060 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.639680 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.640643 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.640806 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.640962 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.641842 (+ 880us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.641943 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":121,"threads_started":1}
I20260926 13:54:12.643128 30516 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3c8244987743d7a0f6ff293b0b6fcd (DEFAULT_TABLE table=test-table [id=0d4ad21371874358be0cab85a0dcc603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:12.644109 30516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3c8244987743d7a0f6ff293b0b6fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:12.646279 30784 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3c8244987743d7a0f6ff293b0b6fcd (DEFAULT_TABLE table=test-table [id=0d4ad21371874358be0cab85a0dcc603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:12.646739 30650 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3c8244987743d7a0f6ff293b0b6fcd (DEFAULT_TABLE table=test-table [id=0d4ad21371874358be0cab85a0dcc603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:12.647459 30784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3c8244987743d7a0f6ff293b0b6fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:12.647835 30650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3c8244987743d7a0f6ff293b0b6fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:12.651558 30867 tablet_bootstrap.cc:492] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: Bootstrap starting.
I20260926 13:54:12.653494 30867 tablet_bootstrap.cc:654] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:12.654630 30867 log.cc:824] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:12.654661 30868 tablet_bootstrap.cc:492] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Bootstrap starting.
I20260926 13:54:12.655628 30869 tablet_bootstrap.cc:492] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Bootstrap starting.
I20260926 13:54:12.656819 30868 tablet_bootstrap.cc:654] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:12.657059 30867 tablet_bootstrap.cc:492] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: No bootstrap required, opened a new log
I20260926 13:54:12.657261 30867 ts_tablet_manager.cc:1397] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:54:12.657812 30869 tablet_bootstrap.cc:654] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:12.658061 30868 log.cc:824] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:12.658969 30869 log.cc:824] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:12.659969 30868 tablet_bootstrap.cc:492] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: No bootstrap required, opened a new log
I20260926 13:54:12.660172 30868 ts_tablet_manager.cc:1397] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:12.660941 30869 tablet_bootstrap.cc:492] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: No bootstrap required, opened a new log
I20260926 13:54:12.661128 30869 ts_tablet_manager.cc:1397] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 13:54:12.661599 30867 raft_consensus.cc:348] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.661921 30867 raft_consensus.cc:374] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:12.662019 30867 raft_consensus.cc:729] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc1076f168e445e9898c773187241cbb, State: Initialized, Role: FOLLOWER
I20260926 13:54:12.662551 30867 consensus_queue.cc:260] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.663789 30867 ts_tablet_manager.cc:1428] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 13:54:12.664029 30868 raft_consensus.cc:348] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.664232 30582 heartbeater.cc:503] Master 127.10.236.254:40049 was elected leader, sending a full tablet report...
I20260926 13:54:12.664321 30868 raft_consensus.cc:374] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:12.664407 30868 raft_consensus.cc:729] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6720307717f4482ca7f05539ed6325b8, State: Initialized, Role: FOLLOWER
I20260926 13:54:12.664856 30868 consensus_queue.cc:260] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.665477 30716 heartbeater.cc:503] Master 127.10.236.254:40049 was elected leader, sending a full tablet report...
I20260926 13:54:12.665825 30868 ts_tablet_manager.cc:1428] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 13:54:12.667573 30869 raft_consensus.cc:348] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.667814 30869 raft_consensus.cc:374] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:12.667882 30869 raft_consensus.cc:729] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc174b40af93439fbb921c973499e259, State: Initialized, Role: FOLLOWER
I20260926 13:54:12.668344 30869 consensus_queue.cc:260] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.669263 30869 ts_tablet_manager.cc:1428] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260926 13:54:12.669370 30850 heartbeater.cc:503] Master 127.10.236.254:40049 was elected leader, sending a full tablet report...
I20260926 13:54:12.707770 30875 raft_consensus.cc:482] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:12.707984 30875 raft_consensus.cc:504] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.709095 30875 leader_election.cc:288] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc1076f168e445e9898c773187241cbb (127.10.236.193:33095), 6720307717f4482ca7f05539ed6325b8 (127.10.236.194:36539)
I20260926 13:54:12.713479 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.709577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:60577 (local address 127.10.236.193:33095)
0926 13:54:12.709742 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.709749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.709767 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:54:12.709824 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.709827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.709878 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.710012 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:12.710267 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.710843 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.712266 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.712865 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.712979 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.713033 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.713312 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.713385 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:54:12.713485 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.709431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:33095 (local address 127.10.236.195:60577)
0926 13:54:12.709573 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.709590 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.709714 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.710023 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.710026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.710042 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:12.710156 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.710161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.710967 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.710971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.712129 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.712142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.713008 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.713012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.713137 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.713171 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.713267 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.713342 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 13:54:12.714259 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.709431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:36539 (local address 127.10.236.195:39763)
0926 13:54:12.710122 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.710136 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:12.710292 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:12.710590 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:12.710594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:12.710612 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:12.710760 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.710766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.711662 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.711667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.712787 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:12.712797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.713925 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.713928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:12.714002 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.714020 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:54:12.714086 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 13:54:12.714152 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":230,"thread_start_us":232,"threads_started":1}
I20260926 13:54:12.714221 30536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3c8244987743d7a0f6ff293b0b6fcd" candidate_uuid: "cc174b40af93439fbb921c973499e259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc1076f168e445e9898c773187241cbb" is_pre_election: true
I20260926 13:54:12.714332 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.709548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:39763 (local address 127.10.236.194:36539)
0926 13:54:12.709714 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.709721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.710231 (+ 510us) server_negotiation.cc:407] Connection header received
0926 13:54:12.710415 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.710420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.710481 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.710584 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:12.710978 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.711544 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.712986 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.713687 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.713810 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.713887 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.714132 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.714209 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 13:54:12.714573 30536 raft_consensus.cc:2454] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc174b40af93439fbb921c973499e259 in term 0.
I20260926 13:54:12.715067 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3c8244987743d7a0f6ff293b0b6fcd" candidate_uuid: "cc174b40af93439fbb921c973499e259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6720307717f4482ca7f05539ed6325b8" is_pre_election: true
I20260926 13:54:12.715291 30670 raft_consensus.cc:2454] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc174b40af93439fbb921c973499e259 in term 0.
I20260926 13:54:12.715771 30739 leader_election.cc:302] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6720307717f4482ca7f05539ed6325b8, cc174b40af93439fbb921c973499e259; no voters:
I20260926 13:54:12.716017 30875 raft_consensus.cc:2781] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:54:12.716099 30875 raft_consensus.cc:482] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:54:12.716142 30875 raft_consensus.cc:3037] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:12.717137 30875 raft_consensus.cc:504] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.717463 30875 leader_election.cc:288] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [CANDIDATE]: Term 1 election: Requested vote from peers dc1076f168e445e9898c773187241cbb (127.10.236.193:33095), 6720307717f4482ca7f05539ed6325b8 (127.10.236.194:36539)
I20260926 13:54:12.717834 30536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3c8244987743d7a0f6ff293b0b6fcd" candidate_uuid: "cc174b40af93439fbb921c973499e259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc1076f168e445e9898c773187241cbb"
I20260926 13:54:12.717823 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3c8244987743d7a0f6ff293b0b6fcd" candidate_uuid: "cc174b40af93439fbb921c973499e259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6720307717f4482ca7f05539ed6325b8"
I20260926 13:54:12.717955 30670 raft_consensus.cc:3037] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:12.717960 30536 raft_consensus.cc:3037] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:12.719291 30536 raft_consensus.cc:2454] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc174b40af93439fbb921c973499e259 in term 1.
I20260926 13:54:12.719293 30670 raft_consensus.cc:2454] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc174b40af93439fbb921c973499e259 in term 1.
I20260926 13:54:12.719626 30740 leader_election.cc:302] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc174b40af93439fbb921c973499e259, dc1076f168e445e9898c773187241cbb; no voters:
I20260926 13:54:12.719780 30875 raft_consensus.cc:2781] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:12.719973 30875 raft_consensus.cc:686] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 1 LEADER]: Becoming Leader. State: Replica: cc174b40af93439fbb921c973499e259, State: Running, Role: LEADER
I20260926 13:54:12.720329 30875 consensus_queue.cc:237] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } }
I20260926 13:54:12.723445 30391 catalog_manager.cc:5495] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc174b40af93439fbb921c973499e259 (127.10.236.195). New cstate: current_term: 1 leader_uuid: "cc174b40af93439fbb921c973499e259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc174b40af93439fbb921c973499e259" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 35315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 } health_report { overall_health: UNKNOWN } } }
I20260926 13:54:12.740863 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.736602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42694 (local address 127.10.236.195:35315)
0926 13:54:12.736743 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:12.736747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:12.736761 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:54:12.736915 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:12.736917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:12.736997 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:12.737086 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:12.737319 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.738069 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.738915 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:12.739204 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:12.739303 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:12.739650 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:12.740276 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:12.740422 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:12.740518 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 13:54:12.740605 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 13:54:12.747089 30536 raft_consensus.cc:1261] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 1 FOLLOWER]: Refusing update from remote peer cc174b40af93439fbb921c973499e259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:12.747089 30670 raft_consensus.cc:1261] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 1 FOLLOWER]: Refusing update from remote peer cc174b40af93439fbb921c973499e259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:12.747853 30875 consensus_queue.cc:1035] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6720307717f4482ca7f05539ed6325b8" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:12.748039 30877 consensus_queue.cc:1035] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc1076f168e445e9898c773187241cbb" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 33095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:12.755604 30880 mvcc.cc:205] Tried to move back new op lower bound from 7333604772844761088 to 7333604772745285632. Current Snapshot: MvccSnapshot[applied={T|T < 7333604772844761088}]
I20260926 13:54:12.756654 30882 mvcc.cc:205] Tried to move back new op lower bound from 7333604772844761088 to 7333604772745285632. Current Snapshot: MvccSnapshot[applied={T|T < 7333604772844761088}]
I20260926 13:54:12.759245 30881 mvcc.cc:205] Tried to move back new op lower bound from 7333604772844761088 to 7333604772745285632. Current Snapshot: MvccSnapshot[applied={T|T < 7333604772844761088}]
I20260926 13:54:12.765437 30390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:12.765637 30390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:12.768399 30390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4048481908754fdb975e4b5532b28150: Sending DeleteTablet for 3 replicas of tablet 4e3c8244987743d7a0f6ff293b0b6fcd
I20260926 13:54:12.769326 30516 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3c8244987743d7a0f6ff293b0b6fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:12 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:44190
I20260926 13:54:12.769883 30890 tablet_replica.cc:330] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb: stopping tablet replica
I20260926 13:54:12.769888 30650 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3c8244987743d7a0f6ff293b0b6fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:12 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:55380
I20260926 13:54:12.770208 30890 raft_consensus.cc:2229] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:12.770394 30891 tablet_replica.cc:330] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: stopping tablet replica
I20260926 13:54:12.770605 30890 raft_consensus.cc:2258] T 4e3c8244987743d7a0f6ff293b0b6fcd P dc1076f168e445e9898c773187241cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:12.770704 30891 raft_consensus.cc:2229] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:12.770686 30784 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3c8244987743d7a0f6ff293b0b6fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:12 UTC) from {username='kudu', principal='kudu/127.10.236.254@KRBTEST.COM'} at 127.0.0.1:42678
I20260926 13:54:12.771013 30891 raft_consensus.cc:2258] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:12.771327 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30453
I20260926 13:54:12.775779 30892 tablet_replica.cc:330] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: stopping tablet replica
I20260926 13:54:12.776141 30892 raft_consensus.cc:2229] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:54:12.776553 30892 raft_consensus.cc:2258] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:12.777448 30891 ts_tablet_manager.cc:1905] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:12.778055 30892 ts_tablet_manager.cc:1905] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:12.780587 30891 ts_tablet_manager.cc:1918] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:12.780831 30891 log.cc:1196] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-1/wal/wals/4e3c8244987743d7a0f6ff293b0b6fcd
I20260926 13:54:12.780923 30892 ts_tablet_manager.cc:1918] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:12.781126 30892 log.cc:1196] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/4e3c8244987743d7a0f6ff293b0b6fcd
I20260926 13:54:12.781342 30891 ts_tablet_manager.cc:1939] T 4e3c8244987743d7a0f6ff293b0b6fcd P 6720307717f4482ca7f05539ed6325b8: Deleting consensus metadata
I20260926 13:54:12.781534 30892 ts_tablet_manager.cc:1939] T 4e3c8244987743d7a0f6ff293b0b6fcd P cc174b40af93439fbb921c973499e259: Deleting consensus metadata
W20260926 13:54:12.781538 30378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:33095 (error 108)
I20260926 13:54:12.782735 30376 catalog_manager.cc:4841] TS cc174b40af93439fbb921c973499e259 (127.10.236.195:35315): tablet 4e3c8244987743d7a0f6ff293b0b6fcd (table test-table [id=0d4ad21371874358be0cab85a0dcc603]) successfully deleted
I20260926 13:54:12.782990 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30587
I20260926 13:54:12.783097 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:12.782755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:33095 (local address 127.0.0.1:44192)
0926 13:54:12.782926 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:12.783002 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 13:54:12.782735 30377 catalog_manager.cc:4841] TS 6720307717f4482ca7f05539ed6325b8 (127.10.236.194:36539): tablet 4e3c8244987743d7a0f6ff293b0b6fcd (table test-table [id=0d4ad21371874358be0cab85a0dcc603]) successfully deleted
W20260926 13:54:12.784237 30378 catalog_manager.cc:4584] TS dc1076f168e445e9898c773187241cbb (127.10.236.193:33095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e3c8244987743d7a0f6ff293b0b6fcd: Network error: Client connection negotiation failed: client connection to 127.10.236.193:33095: connect: Connection refused (error 111)
I20260926 13:54:12.789585 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30721
I20260926 13:54:12.795789 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30359
2026-09-26T13:54:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2291 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T13:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:12Z Disabled control of system clock
I20260926 13:54:12.826333 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33915
--webserver_interface=127.10.236.254
--webserver_port=0
--builtin_ntp_servers=127.10.236.212:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:54:12.943514 30897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:12.943807 30897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:12.943845 30897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:12.947929 30897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:12.947988 30897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:12.948004 30897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:12.948021 30897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:12.948042 30897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:12.953029 30897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:33915
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:33915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:12.954124 30897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:12.955456 30897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:12.964360 30903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:12.964385 30902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:12.964617 30897 server_base.cc:1034] running on GCE node
W20260926 13:54:12.964360 30905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:12.965368 30897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:12.966523 30897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:12.967736 30897 hybrid_clock.cc:648] HybridClock initialized: now 1790430852967700 us; error 47 us; skew 500 ppm
I20260926 13:54:12.969751 30897 webserver.cc:460] Webserver started at http://127.10.236.254:35223/ using document root <none> and password file <none>
I20260926 13:54:12.970398 30897 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:12.970510 30897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:12.970757 30897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:12.972931 30897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "4ffcff767d8a422fac3a6e609dab2948"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "68afc835ed10a12d64b55084397ae3dc"
server_key_iv: "f7d4bc1ef6c4c1c1479efc7421ef87c2"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.973526 30897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "4ffcff767d8a422fac3a6e609dab2948"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "68afc835ed10a12d64b55084397ae3dc"
server_key_iv: "f7d4bc1ef6c4c1c1479efc7421ef87c2"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.977473 30897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 13:54:12.980113 30911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:12.981429 30897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:12.981599 30897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "4ffcff767d8a422fac3a6e609dab2948"
format_stamp: "Formatted at 2026-09-26 13:54:12 on dist-test-slave-r34j"
server_key: "68afc835ed10a12d64b55084397ae3dc"
server_key_iv: "f7d4bc1ef6c4c1c1479efc7421ef87c2"
server_key_version: "encryptionkey@0"
I20260926 13:54:12.981729 30897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:13.015607 30897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:13.016546 30897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:13.016748 30897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:13.024647 30897 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:33915
I20260926 13:54:13.024654 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:33915 every 8 connection(s)
I20260926 13:54:13.025667 30897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:13.029244 30964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:13.033550 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30897
I20260926 13:54:13.033656 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:13.034098 11187 external_mini_cluster.cc:1428] Setting key 4285e21fc73a8b074e9f7aae1350c9f6
I20260926 13:54:13.035794 30964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948: Bootstrap starting.
I20260926 13:54:13.038765 30964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:13.039776 30964 log.cc:824] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:13.041962 30964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948: No bootstrap required, opened a new log
I20260926 13:54:13.046059 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.036080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.10.236.254:33915)
0926 13:54:13.036824 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.036846 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.036892 (+ 46us) server_negotiation.cc:407] Connection header received
0926 13:54:13.037835 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.037861 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.038407 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.038835 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:13.039354 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.040858 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.041831 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.042576 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.042761 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.043678 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:13.043685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:13.043704 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:13.044256 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:13.044335 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:13.044365 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:13.044477 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.044660 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.044958 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260926 13:54:13.047123 30964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffcff767d8a422fac3a6e609dab2948" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33915 } }
I20260926 13:54:13.047349 30964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:13.047415 30964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffcff767d8a422fac3a6e609dab2948, State: Initialized, Role: FOLLOWER
I20260926 13:54:13.047884 30964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffcff767d8a422fac3a6e609dab2948" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33915 } }
I20260926 13:54:13.048012 30964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:13.048084 30964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:13.048192 30964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:13.049299 30964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffcff767d8a422fac3a6e609dab2948" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33915 } }
I20260926 13:54:13.049672 30964 leader_election.cc:302] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ffcff767d8a422fac3a6e609dab2948; no voters:
I20260926 13:54:13.049994 30964 leader_election.cc:288] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:13.050143 30969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:13.050371 30969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [term 1 LEADER]: Becoming Leader. State: Replica: 4ffcff767d8a422fac3a6e609dab2948, State: Running, Role: LEADER
I20260926 13:54:13.050707 30969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffcff767d8a422fac3a6e609dab2948" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33915 } }
I20260926 13:54:13.051321 30964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:13.052414 30971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ffcff767d8a422fac3a6e609dab2948. Latest consensus state: current_term: 1 leader_uuid: "4ffcff767d8a422fac3a6e609dab2948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffcff767d8a422fac3a6e609dab2948" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33915 } } }
I20260926 13:54:13.052436 30970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ffcff767d8a422fac3a6e609dab2948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffcff767d8a422fac3a6e609dab2948" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 33915 } } }
I20260926 13:54:13.052587 30970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:13.052587 30971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:13.052974 30978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:13.056107 30978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:13.061692 30978 catalog_manager.cc:1347] Generated new cluster ID: 4e45730b9d054cffb598365dc8b70340
I20260926 13:54:13.061764 30978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:13.075021 30978 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:13.076102 30978 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:13.082785 30978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948: Generated new TSK 0
I20260926 13:54:13.083467 30978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:13.089349 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.193:0
--local_ip_for_outbound_sockets=127.10.236.193
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33915
--builtin_ntp_servers=127.10.236.212:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:54:13.204252 30988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:13.204511 30988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:13.204598 30988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:13.208770 30988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:13.208834 30988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:13.208890 30988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.193
I20260926 13:54:13.213775 30988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.236.193
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.193
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:13.214931 30988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:13.216434 30988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:13.226305 30994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:13.226317 30996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:13.226316 30993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:13.226392 30988 server_base.cc:1034] running on GCE node
I20260926 13:54:13.226924 30988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:13.227562 30988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:13.228734 30988 hybrid_clock.cc:648] HybridClock initialized: now 1790430853228703 us; error 43 us; skew 500 ppm
I20260926 13:54:13.230609 30988 webserver.cc:460] Webserver started at http://127.10.236.193:36539/ using document root <none> and password file <none>
I20260926 13:54:13.231185 30988 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:13.231245 30988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:13.231463 30988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:13.233582 30988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data/instance:
uuid: "2ebe96e5fab24b8a89093d208dc21829"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "7630aa0687c307453a38849b88cd8a4a"
server_key_iv: "0ecd7d98060405c2be3f49dd83a3a0df"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.234145 30988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance:
uuid: "2ebe96e5fab24b8a89093d208dc21829"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "7630aa0687c307453a38849b88cd8a4a"
server_key_iv: "0ecd7d98060405c2be3f49dd83a3a0df"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.237921 30988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:13.240481 31002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.241747 30988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:54:13.241899 30988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal
uuid: "2ebe96e5fab24b8a89093d208dc21829"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "7630aa0687c307453a38849b88cd8a4a"
server_key_iv: "0ecd7d98060405c2be3f49dd83a3a0df"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.242019 30988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:13.265985 30988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:13.266736 30988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:13.266940 30988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:13.267611 30988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:13.268604 30988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:13.268675 30988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.268741 30988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:13.268785 30988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.277835 30988 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.193:41393
I20260926 13:54:13.277860 31115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.193:41393 every 8 connection(s)
I20260926 13:54:13.278817 30988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/data/info.pb
I20260926 13:54:13.286312 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 30988
I20260926 13:54:13.286444 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-0/wal/instance
I20260926 13:54:13.286785 11187 external_mini_cluster.cc:1428] Setting key 5c1a802cade92d6f1012aeb1a2e7a060
I20260926 13:54:13.288877 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.194:0
--local_ip_for_outbound_sockets=127.10.236.194
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33915
--builtin_ntp_servers=127.10.236.212:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:54:13.289330 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.280788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.193:56669 (local address 127.10.236.254:33915)
0926 13:54:13.280925 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.280929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.281924 (+ 995us) server_negotiation.cc:407] Connection header received
0926 13:54:13.283280 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.283283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.283329 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.283404 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:13.284380 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.284984 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.286308 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.286678 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.286790 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.288129 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:13.288133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:13.288138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:13.288155 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:13.288175 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:13.288184 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:13.288264 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.288963 (+ 699us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.289042 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:54:13.290334 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.281125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33915 (local address 127.10.236.193:56669)
0926 13:54:13.281753 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.281794 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.283052 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.283632 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.283639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.283722 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:13.284240 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.284253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.285123 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.285125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.286185 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.286195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.286857 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.286863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.287035 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.287845 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:13.287847 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:13.287888 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:13.287953 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:13.288765 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:13.288777 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:13.288786 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.288923 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.289245 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260926 13:54:13.291646 31116 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33915
I20260926 13:54:13.291899 31116 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:13.292424 31116 heartbeater.cc:511] Master 127.10.236.254:33915 requested a full tablet report, sending...
I20260926 13:54:13.293979 30928 ts_manager.cc:194] Registered new tserver with Master: 2ebe96e5fab24b8a89093d208dc21829 (127.10.236.193:41393)
I20260926 13:54:13.295161 30928 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.193:56669
W20260926 13:54:13.402734 31119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:13.403066 31119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:13.403137 31119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:13.407171 31119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:13.407265 31119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:13.407358 31119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.194
I20260926 13:54:13.412279 31119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.236.194
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.194
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:13.413403 31119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:13.414794 31119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:13.424929 31124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:13.425006 31119 server_base.cc:1034] running on GCE node
W20260926 13:54:13.424995 31127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:13.424976 31125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:13.425664 31119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:13.426342 31119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:13.427585 31119 hybrid_clock.cc:648] HybridClock initialized: now 1790430853427572 us; error 28 us; skew 500 ppm
I20260926 13:54:13.436472 31119 webserver.cc:460] Webserver started at http://127.10.236.194:33267/ using document root <none> and password file <none>
I20260926 13:54:13.437170 31119 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:13.437278 31119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:13.437539 31119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:13.439986 31119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data/instance:
uuid: "88664aee7c3a4f15a5b0647138dd5382"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "be6dc451234895ff09dcbfb3416bce1e"
server_key_iv: "d06bd2cd7ff9c46e07a3650a5d488119"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.440608 31119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance:
uuid: "88664aee7c3a4f15a5b0647138dd5382"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "be6dc451234895ff09dcbfb3416bce1e"
server_key_iv: "d06bd2cd7ff9c46e07a3650a5d488119"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.444715 31119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 13:54:13.447734 31133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.449033 31119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:13.449185 31119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal
uuid: "88664aee7c3a4f15a5b0647138dd5382"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "be6dc451234895ff09dcbfb3416bce1e"
server_key_iv: "d06bd2cd7ff9c46e07a3650a5d488119"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.449307 31119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:13.466260 31119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:13.466996 31119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:13.467199 31119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:13.467861 31119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:13.468827 31119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:13.468899 31119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.468966 31119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:13.469009 31119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.477926 31119 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.194:46609
I20260926 13:54:13.477950 31246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.194:46609 every 8 connection(s)
I20260926 13:54:13.478830 31119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/data/info.pb
I20260926 13:54:13.486392 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31119
I20260926 13:54:13.486598 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-1/wal/instance
I20260926 13:54:13.487018 11187 external_mini_cluster.cc:1428] Setting key 9447ee7b0962bfd523f695996b41e434
I20260926 13:54:13.488690 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.480861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.194:55243 (local address 127.10.236.254:33915)
0926 13:54:13.481025 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.481030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.482045 (+ 1015us) server_negotiation.cc:407] Connection header received
0926 13:54:13.483472 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.483492 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.483571 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.483672 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:13.484558 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.485089 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.486313 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.486716 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.486816 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.488193 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:13.488197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:13.488200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:13.488213 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:13.488239 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:13.488249 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:13.488326 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.488483 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.488553 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 13:54:13.489295 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.236.195:0
--local_ip_for_outbound_sockets=127.10.236.195
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33915
--builtin_ntp_servers=127.10.236.212:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:54:13.489765 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.481251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33915 (local address 127.10.236.194:55243)
0926 13:54:13.481892 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.481932 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.483262 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.483846 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.483852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.483934 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:13.484410 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.484421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.485212 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.485217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.486152 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.486165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.486887 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.486892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.487127 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.487869 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:13.487871 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:13.487927 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:13.488001 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:13.488328 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:13.488340 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:13.488347 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.488483 (+ 136us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.488771 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260926 13:54:13.490851 31247 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33915
I20260926 13:54:13.491093 31247 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:13.491691 31247 heartbeater.cc:511] Master 127.10.236.254:33915 requested a full tablet report, sending...
I20260926 13:54:13.492761 30928 ts_manager.cc:194] Registered new tserver with Master: 88664aee7c3a4f15a5b0647138dd5382 (127.10.236.194:46609)
I20260926 13:54:13.493639 30928 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.194:55243
W20260926 13:54:13.602391 31250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:13.602664 31250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:13.602703 31250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:13.607031 31250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:13.607111 31250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:13.607177 31250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.236.195
I20260926 13:54:13.612727 31250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.236.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.236.195
--webserver_port=0
--tserver_master_addrs=127.10.236.254:33915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.236.195
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:13.613878 31250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:13.615291 31250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:13.625568 31255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:13.625679 31256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:13.625571 31258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:13.625607 31250 server_base.cc:1034] running on GCE node
I20260926 13:54:13.626189 31250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:13.626876 31250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:13.628055 31250 hybrid_clock.cc:648] HybridClock initialized: now 1790430853628019 us; error 51 us; skew 500 ppm
I20260926 13:54:13.629845 31250 webserver.cc:460] Webserver started at http://127.10.236.195:46561/ using document root <none> and password file <none>
I20260926 13:54:13.630523 31250 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:13.630625 31250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:13.630859 31250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:13.632930 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data/instance:
uuid: "a919498131d645b4ae69c70ee3d84d5a"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "2fd0ea064d5e3c2696fda287c0d0625b"
server_key_iv: "203291427a1ad9eb76babc57eca389f6"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.633497 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance:
uuid: "a919498131d645b4ae69c70ee3d84d5a"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "2fd0ea064d5e3c2696fda287c0d0625b"
server_key_iv: "203291427a1ad9eb76babc57eca389f6"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.637375 31250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:13.639860 31264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.641057 31250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 13:54:13.641211 31250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal
uuid: "a919498131d645b4ae69c70ee3d84d5a"
format_stamp: "Formatted at 2026-09-26 13:54:13 on dist-test-slave-r34j"
server_key: "2fd0ea064d5e3c2696fda287c0d0625b"
server_key_iv: "203291427a1ad9eb76babc57eca389f6"
server_key_version: "encryptionkey@0"
I20260926 13:54:13.641323 31250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:13.664780 31250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:13.665548 31250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:13.665753 31250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:13.666422 31250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:54:13.667388 31250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:54:13.667460 31250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.667547 31250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:54:13.667595 31250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:13.677070 31250 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.195:34109
I20260926 13:54:13.677102 31377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.195:34109 every 8 connection(s)
I20260926 13:54:13.677986 31250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/data/info.pb
I20260926 13:54:13.686698 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31250
I20260926 13:54:13.686838 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal/instance
I20260926 13:54:13.687093 11187 external_mini_cluster.cc:1428] Setting key 05fac02c6774160cbcd788adeafa4871
I20260926 13:54:13.688287 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.679999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:37175 (local address 127.10.236.254:33915)
0926 13:54:13.680154 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.680158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.681300 (+ 1142us) server_negotiation.cc:407] Connection header received
0926 13:54:13.682703 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.682706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.682760 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.682839 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:13.683958 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.684543 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.685909 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.686142 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.686252 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.687705 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:13.687709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:13.687712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:13.687726 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:13.687750 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:13.687760 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:13.687843 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.688027 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.688092 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 13:54:13.689273 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.680413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:33915 (local address 127.10.236.195:37175)
0926 13:54:13.681148 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.681190 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.682494 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.683089 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.683096 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.683184 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:13.683806 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.683823 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.684681 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.684683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.685761 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.685771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.686302 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.686305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.686454 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.687352 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:54:13.687356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:13.687431 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:54:13.687535 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:13.687861 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:13.687870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:13.687876 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.688011 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.688313 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
I20260926 13:54:13.690318 31378 heartbeater.cc:347] Connected to a master server at 127.10.236.254:33915
I20260926 13:54:13.690555 31378 heartbeater.cc:464] Registering TS with master...
I20260926 13:54:13.691074 31378 heartbeater.cc:511] Master 127.10.236.254:33915 requested a full tablet report, sending...
I20260926 13:54:13.692085 30928 ts_manager.cc:194] Registered new tserver with Master: a919498131d645b4ae69c70ee3d84d5a (127.10.236.195:34109)
I20260926 13:54:13.692669 30928 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.236.195:37175
I20260926 13:54:13.701249 11187 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:54:13.707289 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.703330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53284 (local address 127.10.236.254:33915)
0926 13:54:13.703520 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.703524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.703672 (+ 148us) server_negotiation.cc:407] Connection header received
0926 13:54:13.703825 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.703829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.703882 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.703970 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:13.704255 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.704769 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.705650 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.705868 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.705979 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.706828 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:13.706831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:13.706834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:13.706855 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:13.706873 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:13.706881 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:13.706957 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.707079 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.707141 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 13:54:13.710857 30928 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:54:13.714135 30928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:54:13.728819 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.723012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:46609 (local address 127.0.0.1:53840)
0926 13:54:13.723638 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.723653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.723755 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.724435 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.724442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.724468 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:13.724617 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.724624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.726258 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.726261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.727349 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.727360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.728264 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.728268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.728369 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.728404 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.728533 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.728669 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":104,"threads_started":1}
I20260926 13:54:13.728926 31393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.723591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.10.236.194:46609)
0926 13:54:13.723909 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.723923 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.723947 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:54:13.724026 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.724032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.724249 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.724789 (+ 540us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:13.724928 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.726130 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.727528 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.728135 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.728278 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.728467 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.728666 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.728770 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":128,"threads_started":1}
I20260926 13:54:13.729236 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.723218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.10.236.193:41393)
0926 13:54:13.724067 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.724091 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.724117 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:54:13.724197 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.724202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.724395 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.724840 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:13.724989 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.725756 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.727648 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.728424 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.728726 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.728900 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.728950 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.729036 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":101,"threads_started":1}
I20260926 13:54:13.729462 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.723552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.195:34109 (local address 127.0.0.1:41350)
0926 13:54:13.723951 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.723964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.724041 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.724864 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.724867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.724882 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:13.724961 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.724968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.726319 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.726323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.727244 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.727253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.728554 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.728558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.729150 (+ 592us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.729173 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.729257 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.729340 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":59,"thread_start_us":81,"threads_started":1}
I20260926 13:54:13.729700 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.723992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.10.236.195:34109)
0926 13:54:13.724429 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.724440 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.724460 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:54:13.724526 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.724530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.724706 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.724832 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:13.725083 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.726189 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.727368 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.728198 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.728344 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.728522 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.729309 (+ 787us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.729421 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":145,"threads_started":1}
I20260926 13:54:13.729032 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.722813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:41393 (local address 127.0.0.1:34612)
0926 13:54:13.723336 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.723391 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.723549 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.724548 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.724551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.724570 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:13.724685 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.724694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.725877 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.725882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.727526 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.727534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.728563 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.728567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.728735 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.728756 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.728824 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.728891 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":308,"thread_start_us":96,"threads_started":1}
I20260926 13:54:13.732378 31312 tablet_service.cc:1467] Processing CreateTablet for tablet 760f5812199a472b95ea02f7815d9580 (DEFAULT_TABLE table=test-table [id=eabfeae8167f4f889eb49a1036a606e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:13.732402 31181 tablet_service.cc:1467] Processing CreateTablet for tablet 760f5812199a472b95ea02f7815d9580 (DEFAULT_TABLE table=test-table [id=eabfeae8167f4f889eb49a1036a606e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:13.732402 31050 tablet_service.cc:1467] Processing CreateTablet for tablet 760f5812199a472b95ea02f7815d9580 (DEFAULT_TABLE table=test-table [id=eabfeae8167f4f889eb49a1036a606e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:54:13.733376 31312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760f5812199a472b95ea02f7815d9580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:13.733399 31050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760f5812199a472b95ea02f7815d9580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:13.733399 31181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760f5812199a472b95ea02f7815d9580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:13.739773 31395 tablet_bootstrap.cc:492] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Bootstrap starting.
I20260926 13:54:13.741191 31397 tablet_bootstrap.cc:492] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: Bootstrap starting.
I20260926 13:54:13.741194 31396 tablet_bootstrap.cc:492] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: Bootstrap starting.
I20260926 13:54:13.742431 31395 tablet_bootstrap.cc:654] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:13.743062 31397 tablet_bootstrap.cc:654] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:13.743398 31396 tablet_bootstrap.cc:654] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:13.743564 31395 log.cc:824] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:13.743866 31397 log.cc:824] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:13.744264 31396 log.cc:824] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:13.746207 31396 tablet_bootstrap.cc:492] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: No bootstrap required, opened a new log
I20260926 13:54:13.746232 31397 tablet_bootstrap.cc:492] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: No bootstrap required, opened a new log
I20260926 13:54:13.746297 31395 tablet_bootstrap.cc:492] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: No bootstrap required, opened a new log
I20260926 13:54:13.746450 31396 ts_tablet_manager.cc:1397] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:54:13.746446 31397 ts_tablet_manager.cc:1397] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:54:13.746465 31395 ts_tablet_manager.cc:1397] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 13:54:13.749936 31396 raft_consensus.cc:348] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.750218 31396 raft_consensus.cc:374] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:13.750272 31396 raft_consensus.cc:729] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebe96e5fab24b8a89093d208dc21829, State: Initialized, Role: FOLLOWER
I20260926 13:54:13.750726 31396 consensus_queue.cc:260] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.751217 31397 raft_consensus.cc:348] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.751217 31395 raft_consensus.cc:348] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.751565 31395 raft_consensus.cc:374] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:13.751565 31397 raft_consensus.cc:374] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:13.751623 31397 raft_consensus.cc:729] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88664aee7c3a4f15a5b0647138dd5382, State: Initialized, Role: FOLLOWER
I20260926 13:54:13.751633 31395 raft_consensus.cc:729] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a919498131d645b4ae69c70ee3d84d5a, State: Initialized, Role: FOLLOWER
I20260926 13:54:13.751835 31396 ts_tablet_manager.cc:1428] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 13:54:13.752133 31395 consensus_queue.cc:260] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.752218 31116 heartbeater.cc:503] Master 127.10.236.254:33915 was elected leader, sending a full tablet report...
I20260926 13:54:13.752978 31378 heartbeater.cc:503] Master 127.10.236.254:33915 was elected leader, sending a full tablet report...
I20260926 13:54:13.753386 31395 ts_tablet_manager.cc:1428] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 13:54:13.753494 31397 consensus_queue.cc:260] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.754225 31247 heartbeater.cc:503] Master 127.10.236.254:33915 was elected leader, sending a full tablet report...
I20260926 13:54:13.754607 31397 ts_tablet_manager.cc:1428] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260926 13:54:13.780148 31117 tablet.cc:2369] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:54:13.797403 31402 raft_consensus.cc:482] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:54:13.797662 31402 raft_consensus.cc:504] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.798972 31402 leader_election.cc:288] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ebe96e5fab24b8a89093d208dc21829 (127.10.236.193:41393), 88664aee7c3a4f15a5b0647138dd5382 (127.10.236.194:46609)
I20260926 13:54:13.803176 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.799302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:46609 (local address 127.10.236.195:33345)
0926 13:54:13.799748 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.799764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.799897 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.800189 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.800192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.800212 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:13.800324 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.800330 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.801190 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.801196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.802074 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.802083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.802718 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.802722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.802822 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.802848 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.802930 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.803000 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":107,"threads_started":1}
I20260926 13:54:13.803194 31393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.799386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:33345 (local address 127.10.236.194:46609)
0926 13:54:13.799629 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.799639 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.799830 (+ 191us) server_negotiation.cc:407] Connection header received
0926 13:54:13.800024 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.800026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.800103 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.800179 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:13.800447 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.801078 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.802210 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.802597 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.802741 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.802795 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.802953 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.803046 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260926 13:54:13.803334 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.799189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:41393 (local address 127.10.236.195:36931)
0926 13:54:13.799345 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.799361 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:13.799503 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:13.799856 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:13.799860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:13.799881 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:54:13.800017 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.800027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.801103 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.801110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.802139 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:13.802149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.802962 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.802965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:13.803061 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.803081 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:54:13.803151 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:54:13.803223 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 13:54:13.803437 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.799411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.236.195:36931 (local address 127.10.236.193:41393)
0926 13:54:13.799586 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.799591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.799609 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:54:13.799669 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.799672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.799725 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.799820 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:54:13.800135 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.800943 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.802267 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.802753 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.802941 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.803005 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.803191 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.803278 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 13:54:13.804011 31201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760f5812199a472b95ea02f7815d9580" candidate_uuid: "a919498131d645b4ae69c70ee3d84d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88664aee7c3a4f15a5b0647138dd5382" is_pre_election: true
I20260926 13:54:13.804064 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760f5812199a472b95ea02f7815d9580" candidate_uuid: "a919498131d645b4ae69c70ee3d84d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebe96e5fab24b8a89093d208dc21829" is_pre_election: true
I20260926 13:54:13.804344 31070 raft_consensus.cc:2454] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a919498131d645b4ae69c70ee3d84d5a in term 0.
I20260926 13:54:13.804355 31201 raft_consensus.cc:2454] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a919498131d645b4ae69c70ee3d84d5a in term 0.
I20260926 13:54:13.804862 31266 leader_election.cc:302] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88664aee7c3a4f15a5b0647138dd5382, a919498131d645b4ae69c70ee3d84d5a; no voters:
I20260926 13:54:13.805121 31402 raft_consensus.cc:2781] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:54:13.805202 31402 raft_consensus.cc:482] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:54:13.805249 31402 raft_consensus.cc:3037] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:13.806293 31402 raft_consensus.cc:504] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.806640 31402 leader_election.cc:288] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [CANDIDATE]: Term 1 election: Requested vote from peers 2ebe96e5fab24b8a89093d208dc21829 (127.10.236.193:41393), 88664aee7c3a4f15a5b0647138dd5382 (127.10.236.194:46609)
I20260926 13:54:13.807158 31201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760f5812199a472b95ea02f7815d9580" candidate_uuid: "a919498131d645b4ae69c70ee3d84d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88664aee7c3a4f15a5b0647138dd5382"
I20260926 13:54:13.807158 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760f5812199a472b95ea02f7815d9580" candidate_uuid: "a919498131d645b4ae69c70ee3d84d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ebe96e5fab24b8a89093d208dc21829"
I20260926 13:54:13.807327 31070 raft_consensus.cc:3037] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:13.807312 31201 raft_consensus.cc:3037] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:13.808761 31070 raft_consensus.cc:2454] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a919498131d645b4ae69c70ee3d84d5a in term 1.
I20260926 13:54:13.808761 31201 raft_consensus.cc:2454] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a919498131d645b4ae69c70ee3d84d5a in term 1.
I20260926 13:54:13.809121 31266 leader_election.cc:302] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88664aee7c3a4f15a5b0647138dd5382, a919498131d645b4ae69c70ee3d84d5a; no voters:
I20260926 13:54:13.809324 31402 raft_consensus.cc:2781] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:13.809548 31402 raft_consensus.cc:686] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 1 LEADER]: Becoming Leader. State: Replica: a919498131d645b4ae69c70ee3d84d5a, State: Running, Role: LEADER
I20260926 13:54:13.809909 31402 consensus_queue.cc:237] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } }
I20260926 13:54:13.813082 30927 catalog_manager.cc:5495] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to a919498131d645b4ae69c70ee3d84d5a (127.10.236.195). New cstate: current_term: 1 leader_uuid: "a919498131d645b4ae69c70ee3d84d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a919498131d645b4ae69c70ee3d84d5a" member_type: VOTER last_known_addr { host: "127.10.236.195" port: 34109 } health_report { overall_health: HEALTHY } } }
I20260926 13:54:13.828783 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.825091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.10.236.195:34109)
0926 13:54:13.825307 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:13.825311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:13.825324 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:54:13.825403 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:13.825405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:13.825458 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:13.825548 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:13.825794 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.826297 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.827214 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:13.827412 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:13.827552 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:13.827882 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:13.828321 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:13.828437 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:13.828514 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 13:54:13.828561 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260926 13:54:13.834724 31201 raft_consensus.cc:1261] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 1 FOLLOWER]: Refusing update from remote peer a919498131d645b4ae69c70ee3d84d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:13.834776 31070 raft_consensus.cc:1261] T 760f5812199a472b95ea02f7815d9580 P 2ebe96e5fab24b8a89093d208dc21829 [term 1 FOLLOWER]: Refusing update from remote peer a919498131d645b4ae69c70ee3d84d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:13.835449 31405 consensus_queue.cc:1035] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "88664aee7c3a4f15a5b0647138dd5382" member_type: VOTER last_known_addr { host: "127.10.236.194" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:13.835657 31402 consensus_queue.cc:1035] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ebe96e5fab24b8a89093d208dc21829" member_type: VOTER last_known_addr { host: "127.10.236.193" port: 41393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:13.845774 31410 mvcc.cc:205] Tried to move back new op lower bound from 7333604777299394560 to 7333604777208152064. Current Snapshot: MvccSnapshot[applied={T|T < 7333604777299394560}]
I20260926 13:54:13.853787 30926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:13.854040 30926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:54:13.856508 30926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ffcff767d8a422fac3a6e609dab2948: Sending DeleteTablet for 3 replicas of tablet 760f5812199a472b95ea02f7815d9580
I20260926 13:54:13.857350 31181 tablet_service.cc:1514] Processing DeleteTablet for tablet 760f5812199a472b95ea02f7815d9580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:13 UTC) from {username='slave'} at 127.0.0.1:53840
I20260926 13:54:13.857400 31050 tablet_service.cc:1514] Processing DeleteTablet for tablet 760f5812199a472b95ea02f7815d9580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:13 UTC) from {username='slave'} at 127.0.0.1:34612
I20260926 13:54:13.857854 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30988
I20260926 13:54:13.858050 31312 tablet_service.cc:1514] Processing DeleteTablet for tablet 760f5812199a472b95ea02f7815d9580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:54:13 UTC) from {username='slave'} at 127.0.0.1:41350
I20260926 13:54:13.859367 31420 tablet_replica.cc:330] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: stopping tablet replica
I20260926 13:54:13.859723 31420 raft_consensus.cc:2229] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:54:13.860169 31420 raft_consensus.cc:2258] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:54:13.861595 31420 ts_tablet_manager.cc:1905] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:54:13.865417 31420 ts_tablet_manager.cc:1918] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:54:13.865531 31420 log.cc:1196] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Deleting WAL directory at /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790430594225530-11187-0/minicluster-data/ts-2/wal/wals/760f5812199a472b95ea02f7815d9580
I20260926 13:54:13.865863 31420 ts_tablet_manager.cc:1939] T 760f5812199a472b95ea02f7815d9580 P a919498131d645b4ae69c70ee3d84d5a: Deleting consensus metadata
I20260926 13:54:13.865864 31418 tablet_replica.cc:330] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382: stopping tablet replica
I20260926 13:54:13.866323 31418 raft_consensus.cc:2229] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:54:13.866747 31418 raft_consensus.cc:2258] T 760f5812199a472b95ea02f7815d9580 P 88664aee7c3a4f15a5b0647138dd5382 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 13:54:13.867195 30912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.236.193:41393 (error 108)
I20260926 13:54:13.867733 30912 catalog_manager.cc:4841] TS a919498131d645b4ae69c70ee3d84d5a (127.10.236.195:34109): tablet 760f5812199a472b95ea02f7815d9580 (table test-table [id=eabfeae8167f4f889eb49a1036a606e3]) successfully deleted
I20260926 13:54:13.868098 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31119
I20260926 13:54:13.869145 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.868281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.193:41393 (local address 127.0.0.1:34624)
0926 13:54:13.868935 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.869026 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.193:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539}
W20260926 13:54:13.870038 30912 catalog_manager.cc:4584] TS 2ebe96e5fab24b8a89093d208dc21829 (127.10.236.193:41393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 760f5812199a472b95ea02f7815d9580: Network error: Client connection negotiation failed: client connection to 127.10.236.193:41393: connect: Connection refused (error 111)
I20260926 13:54:13.875854 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31250
I20260926 13:54:13.876153 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:13.875864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.194:46609 (local address 127.0.0.1:53854)
0926 13:54:13.876015 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:13.876069 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.194:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 13:54:13.881064 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 30897
2026-09-26T13:54:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1084 ms)
[----------] 6 tests from EncryptionPolicyTest (10703 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:13 dist-test-slave-r34j krb5kdc[31424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:13 dist-test-slave-r34j krb5kdc[31424](info): set up 2 sockets
Sep 26 13:54:13 dist-test-slave-r34j krb5kdc[31424](info): commencing operation
krb5kdc: starting...
W20260926 13:54:14.932350 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:15 dist-test-slave-r34j krb5kdc[31424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:15.067816 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:15.180086 31437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:15.180406 31437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:15.180481 31437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:15.184635 31437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:15.184724 31437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:15.184762 31437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:15.184819 31437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:15.184875 31437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:15.190296 31437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:15.191371 31437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:15.192445 31437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:15.201366 31445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:15.201428 31443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:15.201475 31437 server_base.cc:1034] running on GCE node
W20260926 13:54:15.201582 31442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:15.202385 31437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:15.203610 31437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:15.204860 31437 hybrid_clock.cc:648] HybridClock initialized: now 1790430855204839 us; error 35 us; skew 500 ppm
Sep 26 13:54:15 dist-test-slave-r34j krb5kdc[31424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430855, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:15.207892 31437 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:15.208811 31437 webserver.cc:460] Webserver started at http://127.0.0.1:37007/ using document root <none> and password file <none>
I20260926 13:54:15.209332 31437 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:15.209390 31437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:15.209568 31437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:15.211680 31437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "31cc7cdd17014a7a97f9232fd0e220d0"
format_stamp: "Formatted at 2026-09-26 13:54:15 on dist-test-slave-r34j"
server_key: "61043fde7cbc542f10cd2b6fd8bcda90"
server_key_iv: "945eef07abbe18b218359ebcc02da4cb"
server_key_version: "encryptionkey@0"
I20260926 13:54:15.212213 31437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "31cc7cdd17014a7a97f9232fd0e220d0"
format_stamp: "Formatted at 2026-09-26 13:54:15 on dist-test-slave-r34j"
server_key: "61043fde7cbc542f10cd2b6fd8bcda90"
server_key_iv: "945eef07abbe18b218359ebcc02da4cb"
server_key_version: "encryptionkey@0"
I20260926 13:54:15.216267 31437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 13:54:15.218943 31452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:15.220285 31437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:54:15.220415 31437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "31cc7cdd17014a7a97f9232fd0e220d0"
format_stamp: "Formatted at 2026-09-26 13:54:15 on dist-test-slave-r34j"
server_key: "61043fde7cbc542f10cd2b6fd8bcda90"
server_key_iv: "945eef07abbe18b218359ebcc02da4cb"
server_key_version: "encryptionkey@0"
I20260926 13:54:15.220532 31437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:15.258468 31437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:15.261768 31437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:15.261976 31437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:15.269608 31437 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42233
I20260926 13:54:15.269625 31504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42233 every 8 connection(s)
I20260926 13:54:15.270578 31437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:15.273309 31505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:15.275110 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31437
I20260926 13:54:15.275254 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:15.275563 11187 external_mini_cluster.cc:1428] Setting key 4b2e15f456967e053ae70145f296f0ba
I20260926 13:54:15.279749 31505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0: Bootstrap starting.
I20260926 13:54:15.281929 31505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:15.282685 31505 log.cc:824] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:54:15 dist-test-slave-r34j krb5kdc[31424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:54:15.284652 31505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0: No bootstrap required, opened a new log
I20260926 13:54:15.289462 31505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260926 13:54:15.289743 31505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:15.289801 31505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cc7cdd17014a7a97f9232fd0e220d0, State: Initialized, Role: FOLLOWER
I20260926 13:54:15.290267 31505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260926 13:54:15.290413 31505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:15.290477 31505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:15.290499 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:15.276992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32973 (local address 127.0.0.1:42233)
0926 13:54:15.277444 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:15.277459 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:15.277494 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:54:15.278159 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:15.278176 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:15.278510 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:15.278859 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:15.279731 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:15.280755 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:15.281671 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:15.282197 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:15.282363 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:15.285076 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:15.285082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:15.285103 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:15.285134 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:15.287177 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:15.287812 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:15.287818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:15.287824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:15.287957 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:15.288325 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:15.288329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:15.288331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:15.288850 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:15.289074 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:15.289319 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 13:54:15.289542 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":135,"threads_started":1}
I20260926 13:54:15.290591 31505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:15.291461 31505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260926 13:54:15.291867 31505 leader_election.cc:302] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31cc7cdd17014a7a97f9232fd0e220d0; no voters:
I20260926 13:54:15.292161 31505 leader_election.cc:288] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:15.292269 31510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:15.292507 31510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [term 1 LEADER]: Becoming Leader. State: Replica: 31cc7cdd17014a7a97f9232fd0e220d0, State: Running, Role: LEADER
I20260926 13:54:15.292834 31510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260926 13:54:15.293500 31505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:15.293888 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:15.296375 31511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } } }
I20260926 13:54:15.296617 31511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:15.296893 31512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31cc7cdd17014a7a97f9232fd0e220d0. Latest consensus state: current_term: 1 leader_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31cc7cdd17014a7a97f9232fd0e220d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } } }
I20260926 13:54:15.296980 31512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0 [sys.catalog]: This master's current role is: LEADER
W20260926 13:54:15.299512 31529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:15.299618 31529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:15.299713 31530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:15.300585 31530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:15.306166 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:15.296583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43489 (local address 127.0.0.1:42233)
0926 13:54:15.296793 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:15.296799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:15.297194 (+ 395us) server_negotiation.cc:407] Connection header received
0926 13:54:15.297348 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:15.297351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:15.297394 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:15.297484 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:15.298145 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:15.298705 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:15.299544 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:15.299905 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:15.300010 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:15.301841 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:15.301845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:15.301848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:15.301873 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:15.303602 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:15.304161 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:15.304164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:15.304166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:15.304250 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:15.305115 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:15.305121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:15.305123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:15.305342 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:15.305458 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:15.305894 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 13:54:15.305992 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260926 13:54:15.306797 31530 catalog_manager.cc:1347] Generated new cluster ID: deeec2e3ab9e4ba19cccacdc8edaf491
I20260926 13:54:15.306854 31530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:15.328743 31530 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:15.329792 31530 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:15.335002 31530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31cc7cdd17014a7a97f9232fd0e220d0: Generated new TSK 0
I20260926 13:54:15.335829 31530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:15.341096 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31437
W20260926 13:54:15.346668 31422 connection.cc:535] client connection to 127.0.0.1:42233 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42233): Connection reset by peer (error 104)
2026-09-26T13:54:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:15 dist-test-slave-r34j krb5kdc[31536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:15 dist-test-slave-r34j krb5kdc[31536](info): set up 2 sockets
Sep 26 13:54:15 dist-test-slave-r34j krb5kdc[31536](info): commencing operation
krb5kdc: starting...
W20260926 13:54:16.411710 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:16 dist-test-slave-r34j krb5kdc[31536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:16.536357 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:16.650823 31549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:16.651213 31549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:16.651290 31549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:16.655370 31549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:16.655489 31549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:16.655550 31549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:16.655591 31549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:16.655640 31549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:16.661171 31549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:16.662358 31549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:16.663432 31549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:16.672556 31554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:16.672685 31549 server_base.cc:1034] running on GCE node
W20260926 13:54:16.672616 31555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:16.672643 31557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:16.673372 31549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:16.674655 31549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:16.675833 31549 hybrid_clock.cc:648] HybridClock initialized: now 1790430856675813 us; error 29 us; skew 500 ppm
Sep 26 13:54:16 dist-test-slave-r34j krb5kdc[31536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430856, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:16.678773 31549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:16.679754 31549 webserver.cc:460] Webserver started at http://127.0.0.1:43281/ using document root <none> and password file <none>
I20260926 13:54:16.680305 31549 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:16.680362 31549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:16.680531 31549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:16.682652 31549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "3aa9e60768ad46319476dc7209ebbfcf"
format_stamp: "Formatted at 2026-09-26 13:54:16 on dist-test-slave-r34j"
server_key: "683e8f2ca818e2f5e0fe4e5ed0f09802"
server_key_iv: "85da0e6cb3fe759012f6b64da4e9c958"
server_key_version: "encryptionkey@0"
I20260926 13:54:16.683184 31549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "3aa9e60768ad46319476dc7209ebbfcf"
format_stamp: "Formatted at 2026-09-26 13:54:16 on dist-test-slave-r34j"
server_key: "683e8f2ca818e2f5e0fe4e5ed0f09802"
server_key_iv: "85da0e6cb3fe759012f6b64da4e9c958"
server_key_version: "encryptionkey@0"
I20260926 13:54:16.687054 31549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:16.689718 31564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:16.691216 31549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:54:16.691377 31549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "3aa9e60768ad46319476dc7209ebbfcf"
format_stamp: "Formatted at 2026-09-26 13:54:16 on dist-test-slave-r34j"
server_key: "683e8f2ca818e2f5e0fe4e5ed0f09802"
server_key_iv: "85da0e6cb3fe759012f6b64da4e9c958"
server_key_version: "encryptionkey@0"
I20260926 13:54:16.691534 31549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:16.711687 31549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:16.715042 31549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:16.715265 31549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:16.722252 31549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45713
I20260926 13:54:16.722343 31616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45713 every 8 connection(s)
I20260926 13:54:16.723229 31549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:16.726070 31617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:16.731325 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf: Bootstrap starting.
I20260926 13:54:16.733361 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31549
I20260926 13:54:16.733510 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:16.733825 11187 external_mini_cluster.cc:1428] Setting key 4214a5068232c8dfcad46474fadab228
I20260926 13:54:16.734071 31617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:16.734957 31617 log.cc:824] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:16.736994 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf: No bootstrap required, opened a new log
I20260926 13:54:16.741257 31617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa9e60768ad46319476dc7209ebbfcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45713 } }
I20260926 13:54:16.741472 31617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:16.741504 31617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa9e60768ad46319476dc7209ebbfcf, State: Initialized, Role: FOLLOWER
I20260926 13:54:16.742028 31617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa9e60768ad46319476dc7209ebbfcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45713 } }
I20260926 13:54:16.742198 31617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:16.742280 31617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:16.742401 31617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:16.743000 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:16.734927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34821 (local address 127.0.0.1:45713)
0926 13:54:16.735428 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:16.735444 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:16.735522 (+ 78us) server_negotiation.cc:407] Connection header received
0926 13:54:16.736211 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:16.736229 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:16.736555 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:16.736843 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:16.737235 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:16.738185 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:16.739403 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:16.739934 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:16.740050 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:16.741013 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:16.741019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:16.741031 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:16.741422 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:16.741518 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:16.741555 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:16.741723 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:16.741961 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 13:54:16.742104 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":198,"threads_started":1}
I20260926 13:54:16.743393 31617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa9e60768ad46319476dc7209ebbfcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45713 } }
I20260926 13:54:16.743777 31617 leader_election.cc:302] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3aa9e60768ad46319476dc7209ebbfcf; no voters:
I20260926 13:54:16.744123 31617 leader_election.cc:288] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:16.744210 31622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:16.744444 31622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [term 1 LEADER]: Becoming Leader. State: Replica: 3aa9e60768ad46319476dc7209ebbfcf, State: Running, Role: LEADER
I20260926 13:54:16.744792 31622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa9e60768ad46319476dc7209ebbfcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45713 } }
I20260926 13:54:16.745417 31617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:16.746455 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:16.746713 31624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3aa9e60768ad46319476dc7209ebbfcf. Latest consensus state: current_term: 1 leader_uuid: "3aa9e60768ad46319476dc7209ebbfcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa9e60768ad46319476dc7209ebbfcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45713 } } }
I20260926 13:54:16.746713 31623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3aa9e60768ad46319476dc7209ebbfcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa9e60768ad46319476dc7209ebbfcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45713 } } }
I20260926 13:54:16.746860 31624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:16.746861 31623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf [sys.catalog]: This master's current role is: LEADER
W20260926 13:54:16.750736 31637 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:16.750819 31637 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:16.750878 31632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:16.751791 31632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:16.753181 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:16.749051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36069 (local address 127.0.0.1:45713)
0926 13:54:16.749180 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:16.749184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:16.749197 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:54:16.749328 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:16.749331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:16.749378 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:16.749426 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:16.749787 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:16.750325 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:16.751272 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:16.751557 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:16.751665 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:16.752629 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:16.752632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:16.752635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:16.752649 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:16.752682 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:16.752693 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:16.752790 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:16.752909 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 13:54:16.752943 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:54:16.757936 31632 catalog_manager.cc:1347] Generated new cluster ID: 91afe18548ce42f3836094e62cf7a2b9
I20260926 13:54:16.758031 31632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:16.777119 31632 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:16.778213 31632 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:16.791751 31632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3aa9e60768ad46319476dc7209ebbfcf: Generated new TSK 0
I20260926 13:54:16.792550 31632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:16.798544 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31549
W20260926 13:54:16.804370 31534 connection.cc:535] client connection to 127.0.0.1:45713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45713): Connection reset by peer (error 104)
2026-09-26T13:54:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:16 dist-test-slave-r34j krb5kdc[31648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:16 dist-test-slave-r34j krb5kdc[31648](info): set up 2 sockets
Sep 26 13:54:16 dist-test-slave-r34j krb5kdc[31648](info): commencing operation
krb5kdc: starting...
W20260926 13:54:17.857276 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:17 dist-test-slave-r34j krb5kdc[31648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:17.992236 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:18.107932 31661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:18.108275 31661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:18.108357 31661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:18.112504 31661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:18.112608 31661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:18.112646 31661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:18.112684 31661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:18.112720 31661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:18.118099 31661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:18.119215 31661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:18.120321 31661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:18.129119 31666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:18.129097 31669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:18.129098 31667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:18.129202 31661 server_base.cc:1034] running on GCE node
I20260926 13:54:18.129964 31661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:18.131348 31661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:18.132560 31661 hybrid_clock.cc:648] HybridClock initialized: now 1790430858132554 us; error 40 us; skew 500 ppm
Sep 26 13:54:18 dist-test-slave-r34j krb5kdc[31648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430858, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:18.135584 31661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:18.136547 31661 webserver.cc:460] Webserver started at http://127.0.0.1:41777/ using document root <none> and password file <none>
I20260926 13:54:18.137141 31661 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:18.137236 31661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:18.137457 31661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:18.139559 31661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "aa439ae75c514eaa8c83b92a0498880c"
format_stamp: "Formatted at 2026-09-26 13:54:18 on dist-test-slave-r34j"
server_key: "b4d7cd327960c12c9552f780938c0a2a"
server_key_iv: "3767df598ceb763fe695e967c6c357d1"
server_key_version: "encryptionkey@0"
I20260926 13:54:18.140146 31661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "aa439ae75c514eaa8c83b92a0498880c"
format_stamp: "Formatted at 2026-09-26 13:54:18 on dist-test-slave-r34j"
server_key: "b4d7cd327960c12c9552f780938c0a2a"
server_key_iv: "3767df598ceb763fe695e967c6c357d1"
server_key_version: "encryptionkey@0"
I20260926 13:54:18.143998 31661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:54:18.146485 31676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:18.147851 31661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:54:18.147998 31661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "aa439ae75c514eaa8c83b92a0498880c"
format_stamp: "Formatted at 2026-09-26 13:54:18 on dist-test-slave-r34j"
server_key: "b4d7cd327960c12c9552f780938c0a2a"
server_key_iv: "3767df598ceb763fe695e967c6c357d1"
server_key_version: "encryptionkey@0"
I20260926 13:54:18.148123 31661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:18.152709 31661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:18.156131 31661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:18.156350 31661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:18.163522 31661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41579
I20260926 13:54:18.163550 31728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41579 every 8 connection(s)
I20260926 13:54:18.164470 31661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:18.167200 31729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:18.169117 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31661
I20260926 13:54:18.169271 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:18.169561 11187 external_mini_cluster.cc:1428] Setting key 9efde718534aeb06bf78ddaab9a62000
I20260926 13:54:18.173794 31729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c: Bootstrap starting.
I20260926 13:54:18.174762 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:18.170993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35045 (local address 127.0.0.1:41579)
0926 13:54:18.171429 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:18.171444 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:18.171477 (+ 33us) server_negotiation.cc:407] Connection header received
0926 13:54:18.172171 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:18.172187 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:18.172459 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:18.172788 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:18.173591 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:18.173596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:18.173609 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:18.173619 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:18.173670 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:18.173689 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:18.173837 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:18.173906 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 13:54:18.174066 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":128,"threads_started":1}
I20260926 13:54:18.176124 31729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:18.176923 31729 log.cc:824] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:18.177529 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:18.178825 31729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c: No bootstrap required, opened a new log
I20260926 13:54:18.181114 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:18.179555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33449 (local address 127.0.0.1:41579)
0926 13:54:18.179671 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:18.179675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:18.179834 (+ 159us) server_negotiation.cc:407] Connection header received
0926 13:54:18.179910 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:18.179912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:18.179946 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:18.180024 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:18.180755 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:18.180759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:18.180762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:18.180775 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:18.180802 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:18.180814 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:18.180896 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:18.180960 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 13:54:18.180995 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:54:18.183135 31729 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa439ae75c514eaa8c83b92a0498880c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260926 13:54:18.183344 31729 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:18.183377 31729 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa439ae75c514eaa8c83b92a0498880c, State: Initialized, Role: FOLLOWER
I20260926 13:54:18.183796 31729 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa439ae75c514eaa8c83b92a0498880c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260926 13:54:18.183912 31729 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:18.183959 31729 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:18.184027 31729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:18.184847 31729 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa439ae75c514eaa8c83b92a0498880c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260926 13:54:18.185129 31729 leader_election.cc:302] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa439ae75c514eaa8c83b92a0498880c; no voters:
I20260926 13:54:18.185366 31729 leader_election.cc:288] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:18.185493 31739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:18.185757 31739 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [term 1 LEADER]: Becoming Leader. State: Replica: aa439ae75c514eaa8c83b92a0498880c, State: Running, Role: LEADER
I20260926 13:54:18.186115 31739 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa439ae75c514eaa8c83b92a0498880c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260926 13:54:18.186615 31729 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:18.187727 31741 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa439ae75c514eaa8c83b92a0498880c. Latest consensus state: current_term: 1 leader_uuid: "aa439ae75c514eaa8c83b92a0498880c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa439ae75c514eaa8c83b92a0498880c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } } }
I20260926 13:54:18.187848 31741 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:18.188184 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa439ae75c514eaa8c83b92a0498880c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa439ae75c514eaa8c83b92a0498880c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } } }
I20260926 13:54:18.188335 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:18.188655 31748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:18.191684 31748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:18.196784 31748 catalog_manager.cc:1347] Generated new cluster ID: d180bb3705e64727a90570be0402b346
I20260926 13:54:18.196851 31748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:18.216336 31748 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:18.216728 31748 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:18.225157 31748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa439ae75c514eaa8c83b92a0498880c: Generated new TSK 0
I20260926 13:54:18.225922 31748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:18.232024 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31661
2026-09-26T13:54:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:18 dist-test-slave-r34j krb5kdc[31760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:18 dist-test-slave-r34j krb5kdc[31760](info): set up 2 sockets
Sep 26 13:54:18 dist-test-slave-r34j krb5kdc[31760](info): commencing operation
krb5kdc: starting...
W20260926 13:54:19.287999 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:19 dist-test-slave-r34j krb5kdc[31760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:19.423904 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:19.535895 31773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:19.536175 31773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:19.536216 31773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:19.540333 31773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:19.540401 31773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:19.540421 31773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:19.540437 31773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:19.540452 31773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:19.545917 31773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:19.547091 31773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:19.548169 31773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:19.556953 31779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:19.556975 31781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:19.556964 31778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:19.557315 31773 server_base.cc:1034] running on GCE node
I20260926 13:54:19.557969 31773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:19.559082 31773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:19.560305 31773 hybrid_clock.cc:648] HybridClock initialized: now 1790430859560294 us; error 29 us; skew 500 ppm
Sep 26 13:54:19 dist-test-slave-r34j krb5kdc[31760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:19.568360 31773 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:19.569355 31773 webserver.cc:460] Webserver started at http://127.0.0.1:38413/ using document root <none> and password file <none>
I20260926 13:54:19.569943 31773 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:19.570047 31773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:19.570303 31773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:19.572443 31773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "271c892fb55646ee9ded1254772d0e8a"
format_stamp: "Formatted at 2026-09-26 13:54:19 on dist-test-slave-r34j"
server_key: "169f5159d44cd779f73fe6f7247acf08"
server_key_iv: "bbe69f015d51c8a86f573ac6ac419def"
server_key_version: "encryptionkey@0"
I20260926 13:54:19.573016 31773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "271c892fb55646ee9ded1254772d0e8a"
format_stamp: "Formatted at 2026-09-26 13:54:19 on dist-test-slave-r34j"
server_key: "169f5159d44cd779f73fe6f7247acf08"
server_key_iv: "bbe69f015d51c8a86f573ac6ac419def"
server_key_version: "encryptionkey@0"
I20260926 13:54:19.577088 31773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:54:19.579838 31788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:19.581358 31773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:54:19.581523 31773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "271c892fb55646ee9ded1254772d0e8a"
format_stamp: "Formatted at 2026-09-26 13:54:19 on dist-test-slave-r34j"
server_key: "169f5159d44cd779f73fe6f7247acf08"
server_key_iv: "bbe69f015d51c8a86f573ac6ac419def"
server_key_version: "encryptionkey@0"
I20260926 13:54:19.581670 31773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:19.597864 31773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:19.601181 31773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:19.601405 31773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:19.609215 31773 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42983
I20260926 13:54:19.609257 31840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42983 every 8 connection(s)
I20260926 13:54:19.610280 31773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:19.611215 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31773
I20260926 13:54:19.611366 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:19.611704 11187 external_mini_cluster.cc:1428] Setting key 3cb57b73fe66fd53dd15ccdd0e50e522
I20260926 13:54:19.613934 31841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:19.620085 31841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a: Bootstrap starting.
Sep 26 13:54:19 dist-test-slave-r34j krb5kdc[31760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:54:19.622815 31841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:19.623775 31841 log.cc:824] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:19.626264 31841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a: No bootstrap required, opened a new log
I20260926 13:54:19.628587 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:19.613201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45479 (local address 127.0.0.1:42983)
0926 13:54:19.613702 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:19.613729 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:19.613773 (+ 44us) server_negotiation.cc:407] Connection header received
0926 13:54:19.614583 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:19.614602 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:19.614967 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:19.615317 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:19.616238 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:19.617260 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:19.618201 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:19.618695 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:19.618881 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:19.622095 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:19.622101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:19.622115 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:19.622148 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:19.624286 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:19.624850 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:19.624856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:19.624861 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:19.625029 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:19.625396 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:19.625399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:19.625400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:19.626012 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:19.626252 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:19.627000 (+ 748us) server_negotiation.cc:299] Negotiation successful
0926 13:54:19.627323 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":151,"threads_started":1}
I20260926 13:54:19.630534 31841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271c892fb55646ee9ded1254772d0e8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260926 13:54:19.630873 31841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:19.630950 31841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271c892fb55646ee9ded1254772d0e8a, State: Initialized, Role: FOLLOWER
I20260926 13:54:19.631456 31841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271c892fb55646ee9ded1254772d0e8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260926 13:54:19.631673 31841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:19.631755 31841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:19.631884 31841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:19.633252 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:19.633313 31841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271c892fb55646ee9ded1254772d0e8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260926 13:54:19.633713 31841 leader_election.cc:302] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271c892fb55646ee9ded1254772d0e8a; no voters:
I20260926 13:54:19.634074 31841 leader_election.cc:288] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:19.634232 31846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:19.634490 31846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [term 1 LEADER]: Becoming Leader. State: Replica: 271c892fb55646ee9ded1254772d0e8a, State: Running, Role: LEADER
I20260926 13:54:19.634840 31846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271c892fb55646ee9ded1254772d0e8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260926 13:54:19.635468 31841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:19.636237 31846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "271c892fb55646ee9ded1254772d0e8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271c892fb55646ee9ded1254772d0e8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } } }
I20260926 13:54:19.636296 31847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271c892fb55646ee9ded1254772d0e8a. Latest consensus state: current_term: 1 leader_uuid: "271c892fb55646ee9ded1254772d0e8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271c892fb55646ee9ded1254772d0e8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } } }
I20260926 13:54:19.636466 31847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:19.636613 31846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:19.636870 31860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:19.640254 31860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:19.645718 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:19.635954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39073 (local address 127.0.0.1:42983)
0926 13:54:19.636076 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:19.636080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:19.636313 (+ 233us) server_negotiation.cc:407] Connection header received
0926 13:54:19.636514 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:19.636518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:19.636585 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:19.636646 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:19.637605 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:19.638450 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:19.639581 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:19.639953 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:19.640080 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:19.641793 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:19.641798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:19.641800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:19.641827 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:19.643752 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:19.644371 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:19.644375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:19.644376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:19.644476 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:19.644814 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:19.644818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:19.644820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:19.645008 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:19.645153 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:19.645390 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 13:54:19.645529 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:54:19.647143 31860 catalog_manager.cc:1347] Generated new cluster ID: c42cf3066c4d4cb8a7eda84f75e1b14d
I20260926 13:54:19.647199 31860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:19.664924 31860 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:19.665939 31860 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:19.678196 31860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271c892fb55646ee9ded1254772d0e8a: Generated new TSK 0
I20260926 13:54:19.678965 31860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:19.687919 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31773
2026-09-26T13:54:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:19 dist-test-slave-r34j krb5kdc[31872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:19 dist-test-slave-r34j krb5kdc[31872](info): set up 2 sockets
Sep 26 13:54:19 dist-test-slave-r34j krb5kdc[31872](info): commencing operation
krb5kdc: starting...
W20260926 13:54:20.743872 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:20 dist-test-slave-r34j krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:20.878988 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:20.997241 31885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:20.997534 31885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:20.997578 31885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:21.001734 31885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:21.001801 31885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:21.001821 31885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:21.001837 31885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:21.001852 31885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:21.007261 31885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:21.008440 31885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:21.009528 31885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:21.018980 31893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:21.018980 31890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:21.019089 31885 server_base.cc:1034] running on GCE node
W20260926 13:54:21.018986 31891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:21.020009 31885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:21.021314 31885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:21.022568 31885 hybrid_clock.cc:648] HybridClock initialized: now 1790430861022547 us; error 33 us; skew 500 ppm
Sep 26 13:54:21 dist-test-slave-r34j krb5kdc[31872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430861, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:21.025483 31885 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:21.026420 31885 webserver.cc:460] Webserver started at http://127.0.0.1:34947/ using document root <none> and password file <none>
I20260926 13:54:21.027011 31885 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:21.027114 31885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:21.027348 31885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:21.029531 31885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "a117389ad0544b2b94ac996d1471b395"
format_stamp: "Formatted at 2026-09-26 13:54:21 on dist-test-slave-r34j"
server_key: "141f3c011259025d14fe2acf91850057"
server_key_iv: "dde3b566575e0cb2691b946b4a0727b2"
server_key_version: "encryptionkey@0"
I20260926 13:54:21.030119 31885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "a117389ad0544b2b94ac996d1471b395"
format_stamp: "Formatted at 2026-09-26 13:54:21 on dist-test-slave-r34j"
server_key: "141f3c011259025d14fe2acf91850057"
server_key_iv: "dde3b566575e0cb2691b946b4a0727b2"
server_key_version: "encryptionkey@0"
I20260926 13:54:21.034055 31885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:21.036844 31900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:21.038380 31885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:54:21.038556 31885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "a117389ad0544b2b94ac996d1471b395"
format_stamp: "Formatted at 2026-09-26 13:54:21 on dist-test-slave-r34j"
server_key: "141f3c011259025d14fe2acf91850057"
server_key_iv: "dde3b566575e0cb2691b946b4a0727b2"
server_key_version: "encryptionkey@0"
I20260926 13:54:21.038691 31885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:21.062139 31885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:21.066248 31885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:21.066502 31885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:21.074311 31885 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35871
I20260926 13:54:21.074342 31952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35871 every 8 connection(s)
I20260926 13:54:21.075358 31885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:21.076203 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31885
I20260926 13:54:21.076361 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:21.076778 11187 external_mini_cluster.cc:1428] Setting key 3e35162b387328773ed400e5bbaf2a7d
I20260926 13:54:21.078677 31953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:21.084326 31953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395: Bootstrap starting.
I20260926 13:54:21.086731 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:21.078296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39323 (local address 127.0.0.1:35871)
0926 13:54:21.078832 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:21.078857 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:21.078906 (+ 49us) server_negotiation.cc:407] Connection header received
0926 13:54:21.079716 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:21.079736 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:21.080155 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:21.080529 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:21.080964 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:21.082021 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:21.083030 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:21.083615 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:21.083799 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:21.084812 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:21.084817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:21.084831 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:21.085321 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:21.085380 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:21.085405 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:21.085528 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:21.085704 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 13:54:21.085939 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":151,"threads_started":1}
I20260926 13:54:21.086736 31953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:21.087854 31953 log.cc:824] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:21.089871 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:21.090026 31953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395: No bootstrap required, opened a new log
I20260926 13:54:21.094312 31953 raft_consensus.cc:348] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a117389ad0544b2b94ac996d1471b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260926 13:54:21.094645 31953 raft_consensus.cc:374] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:21.094736 31953 raft_consensus.cc:729] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a117389ad0544b2b94ac996d1471b395, State: Initialized, Role: FOLLOWER
I20260926 13:54:21.095242 31953 consensus_queue.cc:260] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a117389ad0544b2b94ac996d1471b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260926 13:54:21.095422 31953 raft_consensus.cc:388] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:21.095525 31953 raft_consensus.cc:482] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:21.095641 31953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:21.096495 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:21.091999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53601 (local address 127.0.0.1:35871)
0926 13:54:21.092135 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:21.092140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:21.092357 (+ 217us) server_negotiation.cc:407] Connection header received
0926 13:54:21.092459 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:21.092462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:21.092502 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:21.092574 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:21.092932 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:21.093455 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:21.094537 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:21.094890 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:21.095003 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:21.095967 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:21.095971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:21.095973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:21.095985 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:21.096014 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:21.096027 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:21.096108 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:21.096309 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 13:54:21.096370 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:54:21.096932 31953 raft_consensus.cc:504] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a117389ad0544b2b94ac996d1471b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260926 13:54:21.097352 31953 leader_election.cc:302] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a117389ad0544b2b94ac996d1471b395; no voters:
I20260926 13:54:21.097740 31953 leader_election.cc:288] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:21.097851 31963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:21.098104 31963 raft_consensus.cc:686] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [term 1 LEADER]: Becoming Leader. State: Replica: a117389ad0544b2b94ac996d1471b395, State: Running, Role: LEADER
I20260926 13:54:21.098416 31963 consensus_queue.cc:237] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a117389ad0544b2b94ac996d1471b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260926 13:54:21.099028 31953 sys_catalog.cc:564] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:21.100211 31964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a117389ad0544b2b94ac996d1471b395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a117389ad0544b2b94ac996d1471b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } } }
I20260926 13:54:21.100241 31965 sys_catalog.cc:455] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a117389ad0544b2b94ac996d1471b395. Latest consensus state: current_term: 1 leader_uuid: "a117389ad0544b2b94ac996d1471b395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a117389ad0544b2b94ac996d1471b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } } }
I20260926 13:54:21.100431 31965 sys_catalog.cc:458] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:21.100466 31964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:21.101291 31972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:21.104604 31972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:21.110430 31972 catalog_manager.cc:1347] Generated new cluster ID: 1e9c05a7e1ea4407b83ec8f66c9dcf1e
I20260926 13:54:21.110508 31972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:21.121496 31972 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:21.122529 31972 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:21.130195 31972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a117389ad0544b2b94ac996d1471b395: Generated new TSK 0
I20260926 13:54:21.131374 31972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:21.144583 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31885
2026-09-26T13:54:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:21 dist-test-slave-r34j krb5kdc[31984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:21 dist-test-slave-r34j krb5kdc[31984](info): set up 2 sockets
Sep 26 13:54:21 dist-test-slave-r34j krb5kdc[31984](info): commencing operation
krb5kdc: starting...
W20260926 13:54:22.211918 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:22 dist-test-slave-r34j krb5kdc[31984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:22.339663 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:22.451954 31997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:22.452297 31997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:22.452384 31997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:22.456486 31997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:22.456571 31997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:22.456609 31997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:22.456670 31997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:22.456709 31997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:22.462028 31997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:22.463164 31997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:22.464286 31997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:54:22.473095 31997 server_base.cc:1034] running on GCE node
W20260926 13:54:22.473121 32003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:22.473044 32005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:22.473028 32002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:22.473877 31997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:22.475006 31997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:22.476222 31997 hybrid_clock.cc:648] HybridClock initialized: now 1790430862476186 us; error 47 us; skew 500 ppm
Sep 26 13:54:22 dist-test-slave-r34j krb5kdc[31984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:22.479000 31997 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:22.480011 31997 webserver.cc:460] Webserver started at http://127.0.0.1:42371/ using document root <none> and password file <none>
I20260926 13:54:22.480599 31997 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:22.480695 31997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:22.480927 31997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:22.483022 31997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad"
format_stamp: "Formatted at 2026-09-26 13:54:22 on dist-test-slave-r34j"
server_key: "7bf997c618343dbe2703093b928c937e"
server_key_iv: "fffe9f3f5e4e3ed9ecc6b543881e84f0"
server_key_version: "encryptionkey@0"
I20260926 13:54:22.483685 31997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad"
format_stamp: "Formatted at 2026-09-26 13:54:22 on dist-test-slave-r34j"
server_key: "7bf997c618343dbe2703093b928c937e"
server_key_iv: "fffe9f3f5e4e3ed9ecc6b543881e84f0"
server_key_version: "encryptionkey@0"
I20260926 13:54:22.487656 31997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:54:22.490191 32012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:22.491461 31997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:22.491669 31997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad"
format_stamp: "Formatted at 2026-09-26 13:54:22 on dist-test-slave-r34j"
server_key: "7bf997c618343dbe2703093b928c937e"
server_key_iv: "fffe9f3f5e4e3ed9ecc6b543881e84f0"
server_key_version: "encryptionkey@0"
I20260926 13:54:22.491792 31997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:22.496589 31997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:22.499624 31997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:22.499814 31997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:22.507479 31997 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37937
I20260926 13:54:22.507545 32064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37937 every 8 connection(s)
I20260926 13:54:22.508473 31997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:22.511269 32065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:22.516232 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 31997
I20260926 13:54:22.516392 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:22.516683 11187 external_mini_cluster.cc:1428] Setting key 51d3bdec321e17940d292311b8a6b954
I20260926 13:54:22.517431 32065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad: Bootstrap starting.
I20260926 13:54:22.520215 32065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:22.521176 32065 log.cc:824] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:22.522549 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:22.518103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51981 (local address 127.0.0.1:37937)
0926 13:54:22.518570 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:22.518593 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:22.518642 (+ 49us) server_negotiation.cc:407] Connection header received
0926 13:54:22.519509 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:22.519535 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:22.519908 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:22.520293 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:22.521121 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:22.521127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:22.521143 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:22.521156 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:22.521226 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:22.521257 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:22.521366 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:22.521542 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 13:54:22.521723 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":110,"threads_started":1}
I20260926 13:54:22.523414 32065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad: No bootstrap required, opened a new log
I20260926 13:54:22.525475 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:22.527458 32065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260926 13:54:22.527858 32065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:22.527933 32065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a8b6f50e1b84ff0a56f62ffe149c5ad, State: Initialized, Role: FOLLOWER
I20260926 13:54:22.528471 32065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260926 13:54:22.528636 32065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:22.528726 32065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:22.528858 32065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:22.529606 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:22.527739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46097 (local address 127.0.0.1:37937)
0926 13:54:22.527856 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:22.527861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:22.527924 (+ 63us) server_negotiation.cc:407] Connection header received
0926 13:54:22.528155 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:22.528158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:22.528199 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:22.528282 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:22.529248 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:22.529251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:22.529253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:22.529264 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:22.529289 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:22.529299 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:22.529388 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:22.529475 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 13:54:22.529507 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:54:22.530077 32065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260926 13:54:22.530438 32065 leader_election.cc:302] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a8b6f50e1b84ff0a56f62ffe149c5ad; no voters:
I20260926 13:54:22.530759 32065 leader_election.cc:288] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:22.530831 32075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:22.531111 32075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [term 1 LEADER]: Becoming Leader. State: Replica: 5a8b6f50e1b84ff0a56f62ffe149c5ad, State: Running, Role: LEADER
I20260926 13:54:22.531569 32075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260926 13:54:22.532191 32065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:22.533217 32076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } } }
I20260926 13:54:22.533396 32076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:22.533824 32077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a8b6f50e1b84ff0a56f62ffe149c5ad. Latest consensus state: current_term: 1 leader_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b6f50e1b84ff0a56f62ffe149c5ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } } }
I20260926 13:54:22.533926 32077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad [sys.catalog]: This master's current role is: LEADER
W20260926 13:54:22.536880 32090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:22.536952 32090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:22.537038 32084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:22.537724 32084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:22.543534 32084 catalog_manager.cc:1347] Generated new cluster ID: 56d745c8aac64b56a29b21bbeb6b2da9
I20260926 13:54:22.543609 32084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:22.556142 32084 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:22.556470 32084 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:22.564850 32084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a8b6f50e1b84ff0a56f62ffe149c5ad: Generated new TSK 0
I20260926 13:54:22.565711 32084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:22.577720 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 31997
2026-09-26T13:54:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:22 dist-test-slave-r34j krb5kdc[32096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:22 dist-test-slave-r34j krb5kdc[32096](info): set up 2 sockets
Sep 26 13:54:22 dist-test-slave-r34j krb5kdc[32096](info): commencing operation
krb5kdc: starting...
W20260926 13:54:23.631946 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:23 dist-test-slave-r34j krb5kdc[32096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:23.762977 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:23.877342 32109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:23.877748 32109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:23.877849 32109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:23.882325 32109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:23.882441 32109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:23.882479 32109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:23.882510 32109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:23.882551 32109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:23.888229 32109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:23.889426 32109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:23.890542 32109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:23.899791 32115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:23.899793 32114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:23.899775 32117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:23.900250 32109 server_base.cc:1034] running on GCE node
I20260926 13:54:23.900977 32109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:23.902197 32109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:23.903388 32109 hybrid_clock.cc:648] HybridClock initialized: now 1790430863903351 us; error 53 us; skew 500 ppm
Sep 26 13:54:23 dist-test-slave-r34j krb5kdc[32096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:23.906473 32109 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:23.907620 32109 webserver.cc:460] Webserver started at http://127.0.0.1:34185/ using document root <none> and password file <none>
I20260926 13:54:23.908219 32109 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:23.908316 32109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:23.908560 32109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:23.910820 32109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "9fb5c87cd0fd44e1a33196ed14644611"
format_stamp: "Formatted at 2026-09-26 13:54:23 on dist-test-slave-r34j"
server_key: "69a58063454b4ff35fc294212f1c5899"
server_key_iv: "657eeda0be8b532677411e13299ff640"
server_key_version: "encryptionkey@0"
I20260926 13:54:23.911572 32109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "9fb5c87cd0fd44e1a33196ed14644611"
format_stamp: "Formatted at 2026-09-26 13:54:23 on dist-test-slave-r34j"
server_key: "69a58063454b4ff35fc294212f1c5899"
server_key_iv: "657eeda0be8b532677411e13299ff640"
server_key_version: "encryptionkey@0"
I20260926 13:54:23.915755 32109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:54:23.918826 32124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:23.920388 32109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:23.920564 32109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "9fb5c87cd0fd44e1a33196ed14644611"
format_stamp: "Formatted at 2026-09-26 13:54:23 on dist-test-slave-r34j"
server_key: "69a58063454b4ff35fc294212f1c5899"
server_key_iv: "657eeda0be8b532677411e13299ff640"
server_key_version: "encryptionkey@0"
I20260926 13:54:23.920702 32109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:23.936872 32109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:23.940389 32109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:23.940613 32109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:23.949265 32109 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37949
I20260926 13:54:23.949293 32176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37949 every 8 connection(s)
I20260926 13:54:23.950369 32109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:23.953470 32177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:23.959506 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611: Bootstrap starting.
I20260926 13:54:23.960188 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 32109
I20260926 13:54:23.960356 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:23.960696 11187 external_mini_cluster.cc:1428] Setting key 438faa496f6165d975e8be0b053672b3
I20260926 13:54:23.962926 32177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:23.963822 32177 log.cc:824] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:23.966059 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611: No bootstrap required, opened a new log
I20260926 13:54:23.970021 32177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb5c87cd0fd44e1a33196ed14644611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260926 13:54:23.970340 32177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:23.970439 32177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fb5c87cd0fd44e1a33196ed14644611, State: Initialized, Role: FOLLOWER
I20260926 13:54:23.971014 32177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb5c87cd0fd44e1a33196ed14644611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260926 13:54:23.971213 32177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:23.971295 32177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:23.971398 32177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 13:54:23 dist-test-slave-r34j krb5kdc[32096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:54:23.972916 32177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb5c87cd0fd44e1a33196ed14644611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260926 13:54:23.973353 32177 leader_election.cc:302] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fb5c87cd0fd44e1a33196ed14644611; no voters:
I20260926 13:54:23.973886 32177 leader_election.cc:288] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:23.974068 32182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:23.974334 32182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [term 1 LEADER]: Becoming Leader. State: Replica: 9fb5c87cd0fd44e1a33196ed14644611, State: Running, Role: LEADER
I20260926 13:54:23.974776 32182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb5c87cd0fd44e1a33196ed14644611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260926 13:54:23.975446 32177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:23.976850 32183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fb5c87cd0fd44e1a33196ed14644611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb5c87cd0fd44e1a33196ed14644611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } } }
I20260926 13:54:23.976996 32183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:23.977427 32184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fb5c87cd0fd44e1a33196ed14644611. Latest consensus state: current_term: 1 leader_uuid: "9fb5c87cd0fd44e1a33196ed14644611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb5c87cd0fd44e1a33196ed14644611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } } }
I20260926 13:54:23.977607 32184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:23.977938 32191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:23.980235 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:23.962331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:52921 (local address 127.0.0.1:37949)
0926 13:54:23.962857 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:23.962873 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:23.962913 (+ 40us) server_negotiation.cc:407] Connection header received
0926 13:54:23.963977 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:23.964008 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:23.964545 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:23.964992 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:23.965946 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:23.967301 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:23.968394 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:23.968944 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:23.969143 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:23.972741 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:23.972746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:23.972762 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:23.972795 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:23.975403 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:23.976354 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:23.976361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:23.976370 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:23.976530 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:23.977018 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:23.977021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:23.977023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:23.977612 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:23.978720 (+ 1108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:23.978931 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 13:54:23.979153 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":161,"threads_started":1}
I20260926 13:54:23.981298 32191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:23.983839 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:23.987437 32191 catalog_manager.cc:1347] Generated new cluster ID: dfdd7aa8f53c4606995afd869a76f691
I20260926 13:54:23.987545 32191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:23.997838 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:23.986423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:50487 (local address 127.0.0.1:37949)
0926 13:54:23.986549 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:23.986555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:23.987989 (+ 1434us) server_negotiation.cc:407] Connection header received
0926 13:54:23.988131 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:23.988135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:23.988196 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:23.988309 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:23.989226 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:23.990023 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:23.991595 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:23.991968 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:23.992092 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:23.993843 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:23.993848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:23.993851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:23.993883 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:23.995756 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:23.996329 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:23.996333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:23.996335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:23.996436 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:23.996783 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:23.996786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:23.996788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:23.997079 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:23.997252 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:23.997520 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 13:54:23.997630 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:54:24.001371 32191 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:24.002650 32191 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:24.016366 32191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fb5c87cd0fd44e1a33196ed14644611: Generated new TSK 0
I20260926 13:54:24.017362 32191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:24.025463 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 32109
W20260926 13:54:24.031330 32094 connection.cc:535] client connection to 127.0.0.1:37949 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37949): Connection reset by peer (error 104)
2026-09-26T13:54:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:24 dist-test-slave-r34j krb5kdc[32208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:24 dist-test-slave-r34j krb5kdc[32208](info): set up 2 sockets
Sep 26 13:54:24 dist-test-slave-r34j krb5kdc[32208](info): commencing operation
krb5kdc: starting...
W20260926 13:54:25.099885 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:25 dist-test-slave-r34j krb5kdc[32208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:25.231881 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:25.342990 32221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:25.343286 32221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:25.343330 32221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:25.347653 32221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:25.347728 32221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:25.347749 32221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:25.347766 32221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:25.347781 32221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:25.353155 32221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:25.354316 32221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:25.355357 32221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:54:25.363900 32221 server_base.cc:1034] running on GCE node
W20260926 13:54:25.363888 32229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:25.363860 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:25.363862 32227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:25.364892 32221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:25.366040 32221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:25.367237 32221 hybrid_clock.cc:648] HybridClock initialized: now 1790430865367229 us; error 32 us; skew 500 ppm
Sep 26 13:54:25 dist-test-slave-r34j krb5kdc[32208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:25.381162 32221 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:25.382200 32221 webserver.cc:460] Webserver started at http://127.0.0.1:39449/ using document root <none> and password file <none>
I20260926 13:54:25.382813 32221 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:25.382911 32221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:25.383159 32221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:25.385347 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "41b115e33f66463f810df1448b115a07"
format_stamp: "Formatted at 2026-09-26 13:54:25 on dist-test-slave-r34j"
server_key: "46eff5c51b5c291524b4cb3748a68a2a"
server_key_iv: "1c82557380fc18b5f9f04d5521083ce9"
server_key_version: "encryptionkey@0"
I20260926 13:54:25.385939 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "41b115e33f66463f810df1448b115a07"
format_stamp: "Formatted at 2026-09-26 13:54:25 on dist-test-slave-r34j"
server_key: "46eff5c51b5c291524b4cb3748a68a2a"
server_key_iv: "1c82557380fc18b5f9f04d5521083ce9"
server_key_version: "encryptionkey@0"
I20260926 13:54:25.389989 32221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:25.392642 32236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:25.394003 32221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:25.394220 32221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "41b115e33f66463f810df1448b115a07"
format_stamp: "Formatted at 2026-09-26 13:54:25 on dist-test-slave-r34j"
server_key: "46eff5c51b5c291524b4cb3748a68a2a"
server_key_iv: "1c82557380fc18b5f9f04d5521083ce9"
server_key_version: "encryptionkey@0"
I20260926 13:54:25.394348 32221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:25.411305 32221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:25.414558 32221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:25.414779 32221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:25.422235 32221 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33891
I20260926 13:54:25.422250 32288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33891 every 8 connection(s)
I20260926 13:54:25.423408 32221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:25.426502 32289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:25.428594 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 32221
I20260926 13:54:25.428716 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:25.429075 11187 external_mini_cluster.cc:1428] Setting key 6cc5dfef3176033f0e9ee11d628ca000
I20260926 13:54:25.434058 32289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07: Bootstrap starting.
I20260926 13:54:25.436838 32289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:25.438150 32289 log.cc:824] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:25.439713 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:25.430607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:46727 (local address 127.0.0.1:33891)
0926 13:54:25.431106 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:25.431129 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:25.431176 (+ 47us) server_negotiation.cc:407] Connection header received
0926 13:54:25.432032 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:25.432058 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:25.432507 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:25.432895 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:25.433372 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:25.434805 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:25.435790 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:25.436472 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:25.436665 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:25.437551 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:25.437559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:25.437576 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:25.438137 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:25.438209 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:25.438246 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:25.438392 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:25.438639 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 13:54:25.438787 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":126,"threads_started":1}
I20260926 13:54:25.440719 32289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07: No bootstrap required, opened a new log
I20260926 13:54:25.442895 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:25.444705 32289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b115e33f66463f810df1448b115a07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33891 } }
I20260926 13:54:25.444984 32289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:25.445079 32289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41b115e33f66463f810df1448b115a07, State: Initialized, Role: FOLLOWER
I20260926 13:54:25.445574 32289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b115e33f66463f810df1448b115a07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33891 } }
I20260926 13:54:25.445734 32289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:25.445807 32289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:25.445930 32289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:25.446983 32289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b115e33f66463f810df1448b115a07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33891 } }
I20260926 13:54:25.447345 32289 leader_election.cc:302] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41b115e33f66463f810df1448b115a07; no voters:
I20260926 13:54:25.447724 32289 leader_election.cc:288] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:25.447799 32299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:25.448012 32299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [term 1 LEADER]: Becoming Leader. State: Replica: 41b115e33f66463f810df1448b115a07, State: Running, Role: LEADER
I20260926 13:54:25.448354 32299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b115e33f66463f810df1448b115a07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33891 } }
I20260926 13:54:25.449115 32289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:25.449357 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:25.445008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:45231 (local address 127.0.0.1:33891)
0926 13:54:25.445111 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:25.445115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:25.445441 (+ 326us) server_negotiation.cc:407] Connection header received
0926 13:54:25.445577 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:25.445580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:25.445630 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:25.445722 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:25.446027 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:25.446541 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:25.447567 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:25.447822 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:25.447949 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:25.448863 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:25.448866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:25.448868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:25.448880 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:25.448910 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:25.448920 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:25.449020 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:25.449173 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 13:54:25.449234 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 13:54:25.450877 32300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41b115e33f66463f810df1448b115a07. Latest consensus state: current_term: 1 leader_uuid: "41b115e33f66463f810df1448b115a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b115e33f66463f810df1448b115a07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33891 } } }
I20260926 13:54:25.451174 32301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41b115e33f66463f810df1448b115a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b115e33f66463f810df1448b115a07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33891 } } }
I20260926 13:54:25.451654 32301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:25.451320 32300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07 [sys.catalog]: This master's current role is: LEADER
W20260926 13:54:25.453943 32313 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:25.454042 32313 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:25.454160 32315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:25.455024 32315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:25.460695 32315 catalog_manager.cc:1347] Generated new cluster ID: 2581bee478044a69abb219f7527c6614
I20260926 13:54:25.460845 32315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:25.474354 32315 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:25.475370 32315 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:25.489365 32315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41b115e33f66463f810df1448b115a07: Generated new TSK 0
I20260926 13:54:25.490175 32315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:25.500131 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 32221
W20260926 13:54:25.506243 32206 connection.cc:535] client connection to 127.0.0.1:33891 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33891): Connection reset by peer (error 104)
2026-09-26T13:54:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:25 dist-test-slave-r34j krb5kdc[32320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:25 dist-test-slave-r34j krb5kdc[32320](info): set up 2 sockets
Sep 26 13:54:25 dist-test-slave-r34j krb5kdc[32320](info): commencing operation
krb5kdc: starting...
W20260926 13:54:26.555778 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:26 dist-test-slave-r34j krb5kdc[32320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:26.688125 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:26.800263 32333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:26.800580 32333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:26.800653 32333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:26.804804 32333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:26.804898 32333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:26.804935 32333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:26.804996 32333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:26.805035 32333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:26.810428 32333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:26.811635 32333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:26.812721 32333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:26.821779 32339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:26.821798 32341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:26.821939 32333 server_base.cc:1034] running on GCE node
W20260926 13:54:26.821803 32338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:26.822710 32333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:26.823980 32333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:26.825153 32333 hybrid_clock.cc:648] HybridClock initialized: now 1790430866825133 us; error 31 us; skew 500 ppm
Sep 26 13:54:26 dist-test-slave-r34j krb5kdc[32320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:26.828174 32333 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:26.829159 32333 webserver.cc:460] Webserver started at http://127.0.0.1:35159/ using document root <none> and password file <none>
I20260926 13:54:26.829730 32333 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:26.829789 32333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:26.830134 32333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:26.832463 32333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "d4682f946b5d4aeab2a502e3ada99aaa"
format_stamp: "Formatted at 2026-09-26 13:54:26 on dist-test-slave-r34j"
server_key: "25f0927805563f5399b681c1a7cc148b"
server_key_iv: "7f134193c746310c8d721310d4c47c8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:26.833060 32333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "d4682f946b5d4aeab2a502e3ada99aaa"
format_stamp: "Formatted at 2026-09-26 13:54:26 on dist-test-slave-r34j"
server_key: "25f0927805563f5399b681c1a7cc148b"
server_key_iv: "7f134193c746310c8d721310d4c47c8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:26.837160 32333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:26.839887 32348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:26.841202 32333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 13:54:26.841352 32333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "d4682f946b5d4aeab2a502e3ada99aaa"
format_stamp: "Formatted at 2026-09-26 13:54:26 on dist-test-slave-r34j"
server_key: "25f0927805563f5399b681c1a7cc148b"
server_key_iv: "7f134193c746310c8d721310d4c47c8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:26.841477 32333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:26.846369 32333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:26.849522 32333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:26.849725 32333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:26.857247 32333 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35941
I20260926 13:54:26.857264 32400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35941 every 8 connection(s)
I20260926 13:54:26.858219 32333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:26.861043 32401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:26.864437 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 32333
I20260926 13:54:26.864598 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:26.864895 11187 external_mini_cluster.cc:1428] Setting key 0fdab8522f7c1579b39cabeb8de63ea1
I20260926 13:54:26.867063 32401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa: Bootstrap starting.
I20260926 13:54:26.869779 32401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:26.870245 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:26.866288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:44283 (local address 127.0.0.1:35941)
0926 13:54:26.866716 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:26.866732 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:26.866778 (+ 46us) server_negotiation.cc:407] Connection header received
0926 13:54:26.867456 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:26.867475 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:26.867833 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:26.868111 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:26.868971 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:26.868976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:26.868987 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:26.868998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:26.869052 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:26.869071 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:26.869176 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:26.869301 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 13:54:26.869490 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":145,"threads_started":1}
I20260926 13:54:26.870635 32401 log.cc:824] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:26.872836 32401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa: No bootstrap required, opened a new log
I20260926 13:54:26.873219 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:26.876646 32401 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260926 13:54:26.876902 32401 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:26.876984 32401 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4682f946b5d4aeab2a502e3ada99aaa, State: Initialized, Role: FOLLOWER
I20260926 13:54:26.877163 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:26.875291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:54939 (local address 127.0.0.1:35941)
0926 13:54:26.875425 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:26.875429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:26.875708 (+ 279us) server_negotiation.cc:407] Connection header received
0926 13:54:26.875847 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:26.875850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:26.875883 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:26.875974 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:26.876702 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:26.876705 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:26.876708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:26.876720 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:26.876748 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:26.876762 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:26.876860 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:26.876984 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 13:54:26.877027 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:54:26.877480 32401 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260926 13:54:26.877669 32401 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:26.877740 32401 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:26.877843 32401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:26.878854 32401 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260926 13:54:26.879209 32401 leader_election.cc:302] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4682f946b5d4aeab2a502e3ada99aaa; no voters:
I20260926 13:54:26.879550 32401 leader_election.cc:288] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:26.879659 32411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:26.879910 32411 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [term 1 LEADER]: Becoming Leader. State: Replica: d4682f946b5d4aeab2a502e3ada99aaa, State: Running, Role: LEADER
I20260926 13:54:26.880254 32411 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } }
I20260926 13:54:26.880766 32401 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:26.881924 32413 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4682f946b5d4aeab2a502e3ada99aaa. Latest consensus state: current_term: 1 leader_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } } }
I20260926 13:54:26.882057 32413 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:26.882356 32412 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4682f946b5d4aeab2a502e3ada99aaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35941 } } }
I20260926 13:54:26.882537 32412 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:26.882436 32420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:26.885673 32420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:26.891342 32420 catalog_manager.cc:1347] Generated new cluster ID: 3d1daa52dadf439a9eeab8c300154ae0
I20260926 13:54:26.891408 32420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:26.902416 32420 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:26.902810 32420 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:26.909487 32420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4682f946b5d4aeab2a502e3ada99aaa: Generated new TSK 0
I20260926 13:54:26.910424 32420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:26.914295 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 32333
2026-09-26T13:54:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:26 dist-test-slave-r34j krb5kdc[32432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:26 dist-test-slave-r34j krb5kdc[32432](info): set up 2 sockets
Sep 26 13:54:26 dist-test-slave-r34j krb5kdc[32432](info): commencing operation
krb5kdc: starting...
W20260926 13:54:27.975978 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:28 dist-test-slave-r34j krb5kdc[32432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:28.110538 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:28.220310 32445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:28.220582 32445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:28.220628 32445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:28.224619 32445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:28.224674 32445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:28.224694 32445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:28.224710 32445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:28.224725 32445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:28.229907 32445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:28.231050 32445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:28.232053 32445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:28.240834 32453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:28.240774 32450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:28.240774 32451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:28.240873 32445 server_base.cc:1034] running on GCE node
I20260926 13:54:28.241752 32445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:28.242904 32445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:28.244138 32445 hybrid_clock.cc:648] HybridClock initialized: now 1790430868244105 us; error 46 us; skew 500 ppm
Sep 26 13:54:28 dist-test-slave-r34j krb5kdc[32432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:28.247337 32445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:28.248345 32445 webserver.cc:460] Webserver started at http://127.0.0.1:46297/ using document root <none> and password file <none>
I20260926 13:54:28.248862 32445 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:28.248920 32445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:28.249084 32445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:28.251128 32445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "ba2527f77ebf432187871dba4dfafbcb"
format_stamp: "Formatted at 2026-09-26 13:54:28 on dist-test-slave-r34j"
server_key: "9bd3df41c8e7755dfb0bf7b372101410"
server_key_iv: "1c471dfeb5d1c75c51bdd22aade3a26f"
server_key_version: "encryptionkey@0"
I20260926 13:54:28.251708 32445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "ba2527f77ebf432187871dba4dfafbcb"
format_stamp: "Formatted at 2026-09-26 13:54:28 on dist-test-slave-r34j"
server_key: "9bd3df41c8e7755dfb0bf7b372101410"
server_key_iv: "1c471dfeb5d1c75c51bdd22aade3a26f"
server_key_version: "encryptionkey@0"
I20260926 13:54:28.255446 32445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260926 13:54:28.258046 32460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:28.259325 32445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:28.259471 32445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "ba2527f77ebf432187871dba4dfafbcb"
format_stamp: "Formatted at 2026-09-26 13:54:28 on dist-test-slave-r34j"
server_key: "9bd3df41c8e7755dfb0bf7b372101410"
server_key_iv: "1c471dfeb5d1c75c51bdd22aade3a26f"
server_key_version: "encryptionkey@0"
I20260926 13:54:28.259646 32445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:28.280099 32445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:28.283401 32445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:28.283639 32445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:28.291476 32445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33065
I20260926 13:54:28.291522 32512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33065 every 8 connection(s)
I20260926 13:54:28.292495 32445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:28.295331 32513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:28.297096 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 32445
I20260926 13:54:28.297262 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:28.297544 11187 external_mini_cluster.cc:1428] Setting key b1f9f56be2cd5f77d121dd99583a3e3a
I20260926 13:54:28.301928 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb: Bootstrap starting.
I20260926 13:54:28.304316 32513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:28.305042 32513 log.cc:824] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb: Log is configured to *not* fsync() on all Append() calls
Sep 26 13:54:28 dist-test-slave-r34j krb5kdc[32432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:54:28.307348 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb: No bootstrap required, opened a new log
I20260926 13:54:28.311559 32513 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2527f77ebf432187871dba4dfafbcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260926 13:54:28.311856 32513 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:28.311919 32513 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2527f77ebf432187871dba4dfafbcb, State: Initialized, Role: FOLLOWER
I20260926 13:54:28.312459 32513 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2527f77ebf432187871dba4dfafbcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260926 13:54:28.312649 32513 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:28.312741 32513 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:28.312882 32513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:28.313205 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:28.298985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:41957 (local address 127.0.0.1:33065)
0926 13:54:28.299453 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:28.299468 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:28.299605 (+ 137us) server_negotiation.cc:407] Connection header received
0926 13:54:28.300274 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:28.300296 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:28.300649 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:28.301004 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:28.301909 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:28.303207 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:28.304308 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:28.304886 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:28.305088 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:28.308014 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:28.308019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:28.308032 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:28.308059 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:28.309937 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:28.310495 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:28.310501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:28.310506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:28.310619 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:28.310969 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:28.310974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:28.310982 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:28.311386 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:28.311637 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:28.311999 (+ 362us) server_negotiation.cc:299] Negotiation successful
0926 13:54:28.312255 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":146,"threads_started":1}
I20260926 13:54:28.313977 32513 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2527f77ebf432187871dba4dfafbcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260926 13:54:28.314344 32513 leader_election.cc:302] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba2527f77ebf432187871dba4dfafbcb; no voters:
I20260926 13:54:28.314671 32513 leader_election.cc:288] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:28.314834 32518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:28.315346 32518 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [term 1 LEADER]: Becoming Leader. State: Replica: ba2527f77ebf432187871dba4dfafbcb, State: Running, Role: LEADER
I20260926 13:54:28.315708 32518 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2527f77ebf432187871dba4dfafbcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260926 13:54:28.316262 32513 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:28.316310 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:28.317296 32520 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba2527f77ebf432187871dba4dfafbcb. Latest consensus state: current_term: 1 leader_uuid: "ba2527f77ebf432187871dba4dfafbcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2527f77ebf432187871dba4dfafbcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } } }
I20260926 13:54:28.317420 32520 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:28.317778 32519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba2527f77ebf432187871dba4dfafbcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba2527f77ebf432187871dba4dfafbcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } } }
I20260926 13:54:28.317867 32519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:28.318130 32528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:28.321269 32528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:28.327627 32528 catalog_manager.cc:1347] Generated new cluster ID: 17686e3b4888481c8cc51126196659e8
I20260926 13:54:28.327696 32528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:28.331741 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:28.322401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:47277 (local address 127.0.0.1:33065)
0926 13:54:28.322519 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:28.322524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:28.322917 (+ 393us) server_negotiation.cc:407] Connection header received
0926 13:54:28.323044 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:28.323047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:28.323102 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:28.323187 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:28.324066 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:28.324587 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:28.325418 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:28.325775 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:28.325883 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:28.328058 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:28.328062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:28.328065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:28.328091 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:28.329770 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:28.330347 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:28.330350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:28.330352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:28.330435 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:28.330825 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:28.330828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:28.330829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:28.331026 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:28.331165 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:28.331449 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 13:54:28.331576 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:54:28.340546 32528 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:28.341849 32528 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:28.356719 32528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba2527f77ebf432187871dba4dfafbcb: Generated new TSK 0
I20260926 13:54:28.357542 32528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:28.362429 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 32445
W20260926 13:54:28.367890 32430 connection.cc:535] client connection to 127.0.0.1:33065 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33065): Connection reset by peer (error 104)
2026-09-26T13:54:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:28 dist-test-slave-r34j krb5kdc[32544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:28 dist-test-slave-r34j krb5kdc[32544](info): set up 2 sockets
Sep 26 13:54:28 dist-test-slave-r34j krb5kdc[32544](info): commencing operation
krb5kdc: starting...
W20260926 13:54:29.419816 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:29 dist-test-slave-r34j krb5kdc[32544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:29.550580 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:29.671422 32557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:29.671793 32557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:29.671872 32557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:29.676105 32557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:29.676182 32557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:29.676216 32557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:29.676234 32557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:29.676250 32557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:29.681833 32557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:29.683012 32557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:29.684157 32557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:29.693385 32563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:29.693385 32565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:29.693696 32557 server_base.cc:1034] running on GCE node
W20260926 13:54:29.693420 32562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:29.694479 32557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:29.695699 32557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:29.696898 32557 hybrid_clock.cc:648] HybridClock initialized: now 1790430869696885 us; error 26 us; skew 500 ppm
Sep 26 13:54:29 dist-test-slave-r34j krb5kdc[32544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:29.699869 32557 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:29.700816 32557 webserver.cc:460] Webserver started at http://127.0.0.1:40527/ using document root <none> and password file <none>
I20260926 13:54:29.701395 32557 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:29.701488 32557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:29.701732 32557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:29.703912 32557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "86f9350a0f6543208763c44700c0349d"
format_stamp: "Formatted at 2026-09-26 13:54:29 on dist-test-slave-r34j"
server_key: "4ed5ae57dff5d52104054282cd3d3975"
server_key_iv: "df1f16ba93f6b4e9d18d314c81ef25e2"
server_key_version: "encryptionkey@0"
I20260926 13:54:29.704516 32557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "86f9350a0f6543208763c44700c0349d"
format_stamp: "Formatted at 2026-09-26 13:54:29 on dist-test-slave-r34j"
server_key: "4ed5ae57dff5d52104054282cd3d3975"
server_key_iv: "df1f16ba93f6b4e9d18d314c81ef25e2"
server_key_version: "encryptionkey@0"
I20260926 13:54:29.708869 32557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:54:29.711591 32572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:29.713044 32557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:29.713229 32557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "86f9350a0f6543208763c44700c0349d"
format_stamp: "Formatted at 2026-09-26 13:54:29 on dist-test-slave-r34j"
server_key: "4ed5ae57dff5d52104054282cd3d3975"
server_key_iv: "df1f16ba93f6b4e9d18d314c81ef25e2"
server_key_version: "encryptionkey@0"
I20260926 13:54:29.713369 32557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:29.732165 32557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:29.735762 32557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:29.735973 32557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:29.743484 32557 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46099
I20260926 13:54:29.743532 32624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46099 every 8 connection(s)
I20260926 13:54:29.744498 32557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:29.747439 32625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:29.747812 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 32557
I20260926 13:54:29.747977 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:29.748296 11187 external_mini_cluster.cc:1428] Setting key 64ff847df5dfff0b2e2f68a8e717135f
I20260926 13:54:29.754190 32625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d: Bootstrap starting.
I20260926 13:54:29.756667 32625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:29.757422 32625 log.cc:824] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:29.757673 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:29.749705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:45765 (local address 127.0.0.1:46099)
0926 13:54:29.750192 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:29.750212 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:29.750253 (+ 41us) server_negotiation.cc:407] Connection header received
0926 13:54:29.750960 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:29.750979 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:29.751324 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:29.751751 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:29.752149 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:29.753154 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:29.754067 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:29.754613 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:29.754795 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:29.755885 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:29.755890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:29.755903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:29.756311 (+ 408us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:29.756369 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:29.756392 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:29.756539 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:29.756708 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 13:54:29.756852 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":145,"threads_started":1}
I20260926 13:54:29.759369 32625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d: No bootstrap required, opened a new log
I20260926 13:54:29.760785 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:29.763939 32625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f9350a0f6543208763c44700c0349d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260926 13:54:29.764261 32625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:29.764340 32625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f9350a0f6543208763c44700c0349d, State: Initialized, Role: FOLLOWER
I20260926 13:54:29.764817 32625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f9350a0f6543208763c44700c0349d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260926 13:54:29.764940 32625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:29.765028 32625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:29.765139 32625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:29.766348 32625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f9350a0f6543208763c44700c0349d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260926 13:54:29.766786 32625 leader_election.cc:302] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86f9350a0f6543208763c44700c0349d; no voters:
I20260926 13:54:29.767162 32625 leader_election.cc:288] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:29.767266 32635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:29.767531 32635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [term 1 LEADER]: Becoming Leader. State: Replica: 86f9350a0f6543208763c44700c0349d, State: Running, Role: LEADER
I20260926 13:54:29.768047 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:29.763167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:44493 (local address 127.0.0.1:46099)
0926 13:54:29.763318 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:29.763322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:29.763574 (+ 252us) server_negotiation.cc:407] Connection header received
0926 13:54:29.763727 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:29.763730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:29.763771 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:29.763844 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:29.764158 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:29.764885 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:29.766081 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:29.766344 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:29.766434 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:29.767432 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:29.767436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:29.767438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:29.767453 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:29.767506 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:29.767520 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:29.767586 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:29.767871 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 13:54:29.767911 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:54:29.768661 32625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:29.767904 32635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f9350a0f6543208763c44700c0349d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } }
I20260926 13:54:29.770109 32635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86f9350a0f6543208763c44700c0349d. Latest consensus state: current_term: 1 leader_uuid: "86f9350a0f6543208763c44700c0349d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f9350a0f6543208763c44700c0349d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } } }
I20260926 13:54:29.770251 32635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:29.770802 32636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86f9350a0f6543208763c44700c0349d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f9350a0f6543208763c44700c0349d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46099 } } }
I20260926 13:54:29.770912 32636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:29.771178 32645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:29.774272 32645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:29.780440 32645 catalog_manager.cc:1347] Generated new cluster ID: 5855fac0ae8648e9a996e3b4afbc8f8c
I20260926 13:54:29.780520 32645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:29.788429 32645 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:29.789669 32645 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:29.797257 32645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86f9350a0f6543208763c44700c0349d: Generated new TSK 0
I20260926 13:54:29.798133 32645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:29.803745 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 32557
W20260926 13:54:29.808971 32542 connection.cc:535] client connection to 127.0.0.1:46099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46099): Connection reset by peer (error 104)
2026-09-26T13:54:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:29 dist-test-slave-r34j krb5kdc[32656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:29 dist-test-slave-r34j krb5kdc[32656](info): set up 2 sockets
Sep 26 13:54:29 dist-test-slave-r34j krb5kdc[32656](info): commencing operation
krb5kdc: starting...
W20260926 13:54:30.887809 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:30 dist-test-slave-r34j krb5kdc[32656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:54:31.026672 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:31.142715 32669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:31.143006 32669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:31.143054 32669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:31.147722 32669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:31.147814 32669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:31.147835 32669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:31.147853 32669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:31.147871 32669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:31.154150 32669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:31.155304 32669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:31.156374 32669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:54:31.165544 32669 server_base.cc:1034] running on GCE node
W20260926 13:54:31.165503 32677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:31.165493 32675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:31.165495 32674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:31.166379 32669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:31.168056 32669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:31.169286 32669 hybrid_clock.cc:648] HybridClock initialized: now 1790430871169254 us; error 46 us; skew 500 ppm
Sep 26 13:54:31 dist-test-slave-r34j krb5kdc[32656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:31.180347 32669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:54:31.181474 32669 webserver.cc:460] Webserver started at http://127.0.0.1:34469/ using document root <none> and password file <none>
I20260926 13:54:31.182103 32669 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:31.182204 32669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:31.182452 32669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:31.185009 32669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "72376a368db14aa5a59077de47b02a62"
format_stamp: "Formatted at 2026-09-26 13:54:31 on dist-test-slave-r34j"
server_key: "db2f5c6c6f128a3d4c849757f14ae2c4"
server_key_iv: "dd557fc2e3a185b8f9455b18e1f8e427"
server_key_version: "encryptionkey@0"
I20260926 13:54:31.185632 32669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "72376a368db14aa5a59077de47b02a62"
format_stamp: "Formatted at 2026-09-26 13:54:31 on dist-test-slave-r34j"
server_key: "db2f5c6c6f128a3d4c849757f14ae2c4"
server_key_iv: "dd557fc2e3a185b8f9455b18e1f8e427"
server_key_version: "encryptionkey@0"
I20260926 13:54:31.190004 32669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:54:31.192696 32684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:31.194250 32669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:54:31.194512 32669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "72376a368db14aa5a59077de47b02a62"
format_stamp: "Formatted at 2026-09-26 13:54:31 on dist-test-slave-r34j"
server_key: "db2f5c6c6f128a3d4c849757f14ae2c4"
server_key_iv: "dd557fc2e3a185b8f9455b18e1f8e427"
server_key_version: "encryptionkey@0"
I20260926 13:54:31.194705 32669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:31.200305 32669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:31.204005 32669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:31.204226 32669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:31.212815 32669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44757
I20260926 13:54:31.212836 32736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44757 every 8 connection(s)
I20260926 13:54:31.213939 32669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:31.217005 32737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:31.222491 32737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62: Bootstrap starting.
I20260926 13:54:31.223734 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 32669
I20260926 13:54:31.223893 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:31.224189 11187 external_mini_cluster.cc:1428] Setting key f10576464538a01766aebd7ddb60c8ee
I20260926 13:54:31.225431 32737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:31.226219 32737 log.cc:824] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:31.228224 32737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62: No bootstrap required, opened a new log
I20260926 13:54:31.229079 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:31.225254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:54985 (local address 127.0.0.1:44757)
0926 13:54:31.225676 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:31.225690 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:31.225726 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:54:31.226409 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:31.226434 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:31.226718 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:31.227084 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:31.227890 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:31.227894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:31.227906 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:31.227916 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:31.227969 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:31.227988 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:31.228109 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:31.228206 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 13:54:31.228381 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":132,"threads_started":1}
I20260926 13:54:31.232149 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:31.232412 32737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72376a368db14aa5a59077de47b02a62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260926 13:54:31.232626 32737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:31.232682 32737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72376a368db14aa5a59077de47b02a62, State: Initialized, Role: FOLLOWER
I20260926 13:54:31.233114 32737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72376a368db14aa5a59077de47b02a62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260926 13:54:31.233233 32737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:31.233316 32737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:31.233440 32737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:31.234576 32737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72376a368db14aa5a59077de47b02a62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260926 13:54:31.234954 32737 leader_election.cc:302] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72376a368db14aa5a59077de47b02a62; no voters:
I20260926 13:54:31.235333 32737 leader_election.cc:288] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:31.235373 32747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:31.235569 32747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [term 1 LEADER]: Becoming Leader. State: Replica: 72376a368db14aa5a59077de47b02a62, State: Running, Role: LEADER
I20260926 13:54:31.235862 32747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72376a368db14aa5a59077de47b02a62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260926 13:54:31.236595 32737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:31.237066 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:31.234789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.149:46305 (local address 127.0.0.1:44757)
0926 13:54:31.234913 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:31.234917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:31.235251 (+ 334us) server_negotiation.cc:407] Connection header received
0926 13:54:31.235411 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:31.235415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:31.235450 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:31.235562 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:31.236402 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:31.236405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:31.236407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:31.236418 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:31.236444 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:31.236458 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:31.236538 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:31.236850 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 13:54:31.236908 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:54:31.237051 32749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72376a368db14aa5a59077de47b02a62. Latest consensus state: current_term: 1 leader_uuid: "72376a368db14aa5a59077de47b02a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72376a368db14aa5a59077de47b02a62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } } }
I20260926 13:54:31.237098 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72376a368db14aa5a59077de47b02a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72376a368db14aa5a59077de47b02a62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } } }
I20260926 13:54:31.237178 32749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:31.237394 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:31.237917 32752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:31.241297 32752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:31.246718 32752 catalog_manager.cc:1347] Generated new cluster ID: cdcef5866ad34bbd807b6e99ef49958f
I20260926 13:54:31.246804 32752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:31.264475 32752 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:31.264837 32752 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:31.271889 32752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72376a368db14aa5a59077de47b02a62: Generated new TSK 0
I20260926 13:54:31.272625 32752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:31.280716 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 32669
2026-09-26T13:54:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:31 dist-test-slave-r34j krb5kdc[300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:31 dist-test-slave-r34j krb5kdc[300](info): set up 2 sockets
Sep 26 13:54:31 dist-test-slave-r34j krb5kdc[300](info): commencing operation
krb5kdc: starting...
W20260926 13:54:32.344005 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:32 dist-test-slave-r34j krb5kdc[300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:32.479681 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34115
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:34115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:32.599885 313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:32.600176 313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:32.600278 313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:32.605000 313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:32.605070 313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:32.605093 313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:32.605146 313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:32.605182 313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:32.611419 313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:34115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:34115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:32.612685 313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:32.613797 313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:32.623437 318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:32.623667 319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:32.623734 313 server_base.cc:1034] running on GCE node
W20260926 13:54:32.623442 321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:32.624408 313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:32.625679 313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:32.626919 313 hybrid_clock.cc:648] HybridClock initialized: now 1790430872626903 us; error 27 us; skew 500 ppm
Sep 26 13:54:32 dist-test-slave-r34j krb5kdc[300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430872, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:32.629993 313 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:32.630985 313 webserver.cc:460] Webserver started at http://127.10.236.254:41803/ using document root <none> and password file <none>
I20260926 13:54:32.631644 313 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:32.631740 313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:32.631968 313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:32.634135 313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "8bf6742c2499449697c9a8699f1fded0"
format_stamp: "Formatted at 2026-09-26 13:54:32 on dist-test-slave-r34j"
server_key: "0692a19a38967749e5c3e085b140c924"
server_key_iv: "bcd5189ca710782fe5bf86d208097882"
server_key_version: "encryptionkey@0"
I20260926 13:54:32.634742 313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "8bf6742c2499449697c9a8699f1fded0"
format_stamp: "Formatted at 2026-09-26 13:54:32 on dist-test-slave-r34j"
server_key: "0692a19a38967749e5c3e085b140c924"
server_key_iv: "bcd5189ca710782fe5bf86d208097882"
server_key_version: "encryptionkey@0"
I20260926 13:54:32.638893 313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:32.641688 328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:32.643221 313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:54:32.643442 313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "8bf6742c2499449697c9a8699f1fded0"
format_stamp: "Formatted at 2026-09-26 13:54:32 on dist-test-slave-r34j"
server_key: "0692a19a38967749e5c3e085b140c924"
server_key_iv: "bcd5189ca710782fe5bf86d208097882"
server_key_version: "encryptionkey@0"
I20260926 13:54:32.643627 313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:32.665611 313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:32.668859 313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:32.669087 313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:32.676311 313 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:34115
I20260926 13:54:32.676332 380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:34115 every 8 connection(s)
I20260926 13:54:32.677380 313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:32.680213 381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:32.685644 381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0: Bootstrap starting.
I20260926 13:54:32.686947 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 313
I20260926 13:54:32.687173 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:32.687463 11187 external_mini_cluster.cc:1428] Setting key 2cb88bb012bc5d63cfe9caaf9b6ae30e
I20260926 13:54:32.688340 381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:32.689194 381 log.cc:824] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:32.691138 381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0: No bootstrap required, opened a new log
I20260926 13:54:32.694651 381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf6742c2499449697c9a8699f1fded0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34115 } }
I20260926 13:54:32.694967 381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:32.695051 381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6742c2499449697c9a8699f1fded0, State: Initialized, Role: FOLLOWER
I20260926 13:54:32.695574 381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf6742c2499449697c9a8699f1fded0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34115 } }
I20260926 13:54:32.695736 381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:32.695806 381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:32.695904 381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 13:54:32 dist-test-slave-r34j krb5kdc[300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:32.697162 381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf6742c2499449697c9a8699f1fded0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34115 } }
I20260926 13:54:32.697525 381 leader_election.cc:302] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bf6742c2499449697c9a8699f1fded0; no voters:
I20260926 13:54:32.697909 381 leader_election.cc:288] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:32.698041 386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:32.698369 386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6742c2499449697c9a8699f1fded0, State: Running, Role: LEADER
I20260926 13:54:32.698740 386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf6742c2499449697c9a8699f1fded0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34115 } }
I20260926 13:54:32.699270 381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:32.700086 387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bf6742c2499449697c9a8699f1fded0. Latest consensus state: current_term: 1 leader_uuid: "8bf6742c2499449697c9a8699f1fded0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf6742c2499449697c9a8699f1fded0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34115 } } }
I20260926 13:54:32.700222 387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:32.700495 388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bf6742c2499449697c9a8699f1fded0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf6742c2499449697c9a8699f1fded0" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 34115 } } }
I20260926 13:54:32.700587 388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:32.701026 395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:32.704051 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:32.688650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48583 (local address 127.10.236.254:34115)
0926 13:54:32.689112 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:32.689132 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:32.689169 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:54:32.690096 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:32.690116 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:32.690447 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:32.690802 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:32.691662 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:32.692630 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:32.693510 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:32.693994 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:32.694174 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:32.697451 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:32.697456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:32.697470 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:32.697498 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:32.699831 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:32.700576 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:32.700585 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:32.700592 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:32.700747 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:32.701192 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:32.701196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:32.701201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:32.701794 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:32.702011 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:32.702842 (+ 831us) server_negotiation.cc:299] Negotiation successful
0926 13:54:32.703072 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260926 13:54:32.704296 395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:32.707190 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:32.710944 395 catalog_manager.cc:1347] Generated new cluster ID: 013b6867146842b5aff656bf23694e05
I20260926 13:54:32.711014 395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:32.718407 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:32.709378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37495 (local address 127.10.236.254:34115)
0926 13:54:32.709488 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:32.709493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:32.709730 (+ 237us) server_negotiation.cc:407] Connection header received
0926 13:54:32.709867 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:32.709870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:32.709925 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:32.710009 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:32.710713 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:32.711473 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:32.712654 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:32.713013 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:32.713110 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:32.714903 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:32.714907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:32.714910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:32.714935 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:32.716575 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:32.717104 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:32.717107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:32.717109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:32.717190 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:32.717549 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:32.717552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:32.717554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:32.717751 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:32.717875 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:32.718158 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 13:54:32.718253 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:54:32.731551 395 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:32.732882 395 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:32.742269 395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bf6742c2499449697c9a8699f1fded0: Generated new TSK 0
I20260926 13:54:32.743175 395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:32.756055 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 313
W20260926 13:54:32.761441 32766 connection.cc:535] client connection to 127.10.236.254:34115 recv error: Network error: failed to read from TLS socket (remote: 127.10.236.254:34115): Connection reset by peer (error 104)
2026-09-26T13:54:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:32 dist-test-slave-r34j krb5kdc[412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:32 dist-test-slave-r34j krb5kdc[412](info): set up 2 sockets
Sep 26 13:54:32 dist-test-slave-r34j krb5kdc[412](info): commencing operation
krb5kdc: starting...
W20260926 13:54:33.811874 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:33 dist-test-slave-r34j krb5kdc[412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:33.947956 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:41499
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:41499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:34.063156 425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:34.063476 425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:34.063587 425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:34.068027 425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:34.068107 425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:34.068127 425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:34.068145 425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:34.068161 425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:34.073928 425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:41499
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:41499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:34.075006 425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:34.076013 425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:34.084707 433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:34.084736 431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:34.084848 430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:34.084934 425 server_base.cc:1034] running on GCE node
I20260926 13:54:34.085678 425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:34.086829 425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:34.088006 425 hybrid_clock.cc:648] HybridClock initialized: now 1790430874087980 us; error 38 us; skew 500 ppm
Sep 26 13:54:34 dist-test-slave-r34j krb5kdc[412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430874, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:34.091233 425 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:34.092197 425 webserver.cc:460] Webserver started at http://127.10.236.254:37081/ using document root <none> and password file <none>
I20260926 13:54:34.092765 425 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:34.092826 425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:34.093041 425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:34.095084 425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "f033a14bd8364cafacf1d1b8beba42c2"
format_stamp: "Formatted at 2026-09-26 13:54:34 on dist-test-slave-r34j"
server_key: "8537f8243857c913cfb6aea290b343d3"
server_key_iv: "f3c51033ab2b7dac0b25360a1d6aa641"
server_key_version: "encryptionkey@0"
I20260926 13:54:34.095702 425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "f033a14bd8364cafacf1d1b8beba42c2"
format_stamp: "Formatted at 2026-09-26 13:54:34 on dist-test-slave-r34j"
server_key: "8537f8243857c913cfb6aea290b343d3"
server_key_iv: "f3c51033ab2b7dac0b25360a1d6aa641"
server_key_version: "encryptionkey@0"
I20260926 13:54:34.099404 425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:34.101953 440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:34.103344 425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:34.103543 425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "f033a14bd8364cafacf1d1b8beba42c2"
format_stamp: "Formatted at 2026-09-26 13:54:34 on dist-test-slave-r34j"
server_key: "8537f8243857c913cfb6aea290b343d3"
server_key_iv: "f3c51033ab2b7dac0b25360a1d6aa641"
server_key_version: "encryptionkey@0"
I20260926 13:54:34.103683 425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:34.129841 425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:34.133272 425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:34.133504 425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:34.141793 425 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:41499
I20260926 13:54:34.141815 492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:41499 every 8 connection(s)
I20260926 13:54:34.142829 425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:34.145160 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 425
I20260926 13:54:34.145342 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:34.145646 11187 external_mini_cluster.cc:1428] Setting key af1dd20e127de339e59c8488ba9969f9
I20260926 13:54:34.145946 493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:34.152945 493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2: Bootstrap starting.
I20260926 13:54:34.154891 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:34.147003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44601 (local address 127.10.236.254:41499)
0926 13:54:34.147431 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:34.147447 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:34.147523 (+ 76us) server_negotiation.cc:407] Connection header received
0926 13:54:34.148180 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:34.148196 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:34.148526 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:34.148803 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:34.149133 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:34.150127 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:34.151363 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:34.151966 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:34.152119 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:34.153128 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:34.153134 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:34.153147 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:34.153546 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:34.153616 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:34.153640 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:34.153770 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:34.153971 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 13:54:34.154134 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":149,"threads_started":1}
I20260926 13:54:34.155380 493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:34.156226 493 log.cc:824] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:34.158217 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:34.158599 493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2: No bootstrap required, opened a new log
I20260926 13:54:34.162576 493 raft_consensus.cc:348] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f033a14bd8364cafacf1d1b8beba42c2" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 41499 } }
I20260926 13:54:34.162833 493 raft_consensus.cc:374] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:34.162931 493 raft_consensus.cc:729] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f033a14bd8364cafacf1d1b8beba42c2, State: Initialized, Role: FOLLOWER
I20260926 13:54:34.163462 493 consensus_queue.cc:260] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f033a14bd8364cafacf1d1b8beba42c2" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 41499 } }
I20260926 13:54:34.163715 493 raft_consensus.cc:388] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:34.163794 493 raft_consensus.cc:482] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:34.163934 493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:34.164438 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:34.160346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45071 (local address 127.10.236.254:41499)
0926 13:54:34.160472 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:34.160477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:34.160649 (+ 172us) server_negotiation.cc:407] Connection header received
0926 13:54:34.160839 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:34.160842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:34.160881 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:34.160964 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:34.161272 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:34.161773 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:34.162592 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:34.162826 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:34.162991 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:34.163933 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:34.163937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:34.163941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:34.163954 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:34.163985 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:34.163996 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:34.164108 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:34.164262 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 13:54:34.164302 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:54:34.165231 493 raft_consensus.cc:504] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f033a14bd8364cafacf1d1b8beba42c2" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 41499 } }
I20260926 13:54:34.165617 493 leader_election.cc:302] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f033a14bd8364cafacf1d1b8beba42c2; no voters:
I20260926 13:54:34.165898 493 leader_election.cc:288] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:34.166028 503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:34.166249 503 raft_consensus.cc:686] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [term 1 LEADER]: Becoming Leader. State: Replica: f033a14bd8364cafacf1d1b8beba42c2, State: Running, Role: LEADER
I20260926 13:54:34.166553 503 consensus_queue.cc:237] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f033a14bd8364cafacf1d1b8beba42c2" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 41499 } }
I20260926 13:54:34.167183 493 sys_catalog.cc:564] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:34.168203 504 sys_catalog.cc:455] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f033a14bd8364cafacf1d1b8beba42c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f033a14bd8364cafacf1d1b8beba42c2" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 41499 } } }
I20260926 13:54:34.168365 504 sys_catalog.cc:458] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:34.168700 505 sys_catalog.cc:455] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f033a14bd8364cafacf1d1b8beba42c2. Latest consensus state: current_term: 1 leader_uuid: "f033a14bd8364cafacf1d1b8beba42c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f033a14bd8364cafacf1d1b8beba42c2" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 41499 } } }
I20260926 13:54:34.168803 505 sys_catalog.cc:458] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:34.169049 512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:34.172284 512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:34.177843 512 catalog_manager.cc:1347] Generated new cluster ID: ef21231ff1ec43179ef7eb09d5581498
I20260926 13:54:34.177915 512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:34.201287 512 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:34.202683 512 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:34.209978 512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f033a14bd8364cafacf1d1b8beba42c2: Generated new TSK 0
I20260926 13:54:34.210855 512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:34.215296 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 425
W20260926 13:54:34.220721 410 connection.cc:535] client connection to 127.10.236.254:41499 recv error: Network error: failed to read from TLS socket (remote: 127.10.236.254:41499): Connection reset by peer (error 104)
2026-09-26T13:54:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:34 dist-test-slave-r34j krb5kdc[524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:34 dist-test-slave-r34j krb5kdc[524](info): set up 2 sockets
Sep 26 13:54:34 dist-test-slave-r34j krb5kdc[524](info): commencing operation
krb5kdc: starting...
W20260926 13:54:35.307988 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:35 dist-test-slave-r34j krb5kdc[524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:35.444438 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43971
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:43971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:35.560405 537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:35.560730 537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:35.560777 537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:35.564985 537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:35.565068 537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:35.565092 537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:35.565112 537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:35.565176 537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:35.570791 537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:43971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:43971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:35.571996 537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:35.573061 537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:35.581995 543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:35.581995 545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:35.582108 537 server_base.cc:1034] running on GCE node
W20260926 13:54:35.582059 542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:35.582868 537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:35.584041 537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:35.585220 537 hybrid_clock.cc:648] HybridClock initialized: now 1790430875585188 us; error 43 us; skew 500 ppm
Sep 26 13:54:35 dist-test-slave-r34j krb5kdc[524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430875, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:35.588284 537 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:35.589154 537 webserver.cc:460] Webserver started at http://127.10.236.254:42199/ using document root <none> and password file <none>
I20260926 13:54:35.589756 537 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:35.589848 537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:35.590082 537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:35.592249 537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "c9bcffb6e7ef409b83a19dbcf597755f"
format_stamp: "Formatted at 2026-09-26 13:54:35 on dist-test-slave-r34j"
server_key: "6869283b405d4e9abb88ea50f3618f4a"
server_key_iv: "fa7fbb2248ebd83f918fb060416ca376"
server_key_version: "encryptionkey@0"
I20260926 13:54:35.592811 537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "c9bcffb6e7ef409b83a19dbcf597755f"
format_stamp: "Formatted at 2026-09-26 13:54:35 on dist-test-slave-r34j"
server_key: "6869283b405d4e9abb88ea50f3618f4a"
server_key_iv: "fa7fbb2248ebd83f918fb060416ca376"
server_key_version: "encryptionkey@0"
I20260926 13:54:35.596575 537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 13:54:35.599141 552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:35.600565 537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:35.600721 537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "c9bcffb6e7ef409b83a19dbcf597755f"
format_stamp: "Formatted at 2026-09-26 13:54:35 on dist-test-slave-r34j"
server_key: "6869283b405d4e9abb88ea50f3618f4a"
server_key_iv: "fa7fbb2248ebd83f918fb060416ca376"
server_key_version: "encryptionkey@0"
I20260926 13:54:35.600840 537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:35.605527 537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:35.608901 537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:35.609110 537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:35.616541 537 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:43971
I20260926 13:54:35.616588 604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:43971 every 8 connection(s)
I20260926 13:54:35.617575 537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:35.620530 605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:35.621345 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 537
I20260926 13:54:35.621495 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:35.621814 11187 external_mini_cluster.cc:1428] Setting key 424302116a7764b091a2c07ad94ba560
I20260926 13:54:35.627677 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:35.623261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46191 (local address 127.10.236.254:43971)
0926 13:54:35.623745 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:35.623765 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:35.623812 (+ 47us) server_negotiation.cc:407] Connection header received
0926 13:54:35.624556 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:35.624579 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:35.624941 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:35.625321 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:35.626147 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:35.626152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:35.626168 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:35.626181 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:35.626244 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:35.626272 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:35.626413 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:35.626563 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 13:54:35.626720 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":141,"threads_started":1}
I20260926 13:54:35.628639 605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f: Bootstrap starting.
I20260926 13:54:35.630657 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:35.631109 605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:35.631889 605 log.cc:824] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:35.634037 605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f: No bootstrap required, opened a new log
I20260926 13:54:35.634768 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:35.632992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52579 (local address 127.10.236.254:43971)
0926 13:54:35.633125 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:35.633131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:35.633309 (+ 178us) server_negotiation.cc:407] Connection header received
0926 13:54:35.633453 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:35.633456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:35.633497 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:35.633583 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:35.634361 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:35.634364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:35.634366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:35.634377 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:35.634402 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:35.634413 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:35.634482 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:35.634591 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 13:54:35.634621 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:54:35.637787 605 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43971 } }
I20260926 13:54:35.637998 605 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:35.638072 605 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9bcffb6e7ef409b83a19dbcf597755f, State: Initialized, Role: FOLLOWER
I20260926 13:54:35.638509 605 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43971 } }
I20260926 13:54:35.638634 605 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:35.638718 605 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:35.638834 605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:35.639806 605 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43971 } }
I20260926 13:54:35.640141 605 leader_election.cc:302] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9bcffb6e7ef409b83a19dbcf597755f; no voters:
I20260926 13:54:35.640435 605 leader_election.cc:288] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:35.640560 615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:35.640805 615 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [term 1 LEADER]: Becoming Leader. State: Replica: c9bcffb6e7ef409b83a19dbcf597755f, State: Running, Role: LEADER
I20260926 13:54:35.641155 615 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43971 } }
I20260926 13:54:35.641776 605 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:35.642930 617 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9bcffb6e7ef409b83a19dbcf597755f. Latest consensus state: current_term: 1 leader_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43971 } } }
I20260926 13:54:35.643115 617 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:35.643302 616 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bcffb6e7ef409b83a19dbcf597755f" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 43971 } } }
I20260926 13:54:35.643395 616 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:35.644184 624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:35.646787 624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:35.652673 624 catalog_manager.cc:1347] Generated new cluster ID: a01e36b6a6a14f37b5d245c4269c6f53
I20260926 13:54:35.652748 624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:35.670338 624 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:35.670768 624 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:35.695930 624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9bcffb6e7ef409b83a19dbcf597755f: Generated new TSK 0
I20260926 13:54:35.696844 624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:35.708664 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 537
2026-09-26T13:54:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:35 dist-test-slave-r34j krb5kdc[636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:35 dist-test-slave-r34j krb5kdc[636](info): set up 2 sockets
Sep 26 13:54:35 dist-test-slave-r34j krb5kdc[636](info): commencing operation
krb5kdc: starting...
W20260926 13:54:36.759905 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:36 dist-test-slave-r34j krb5kdc[636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:36.891727 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:42751
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:37.003330 649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:37.003746 649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:37.003836 649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:37.007987 649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:37.008090 649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:37.008122 649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:37.008162 649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:37.008208 649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:37.013707 649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:42751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:42751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:37.014885 649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:37.015982 649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:37.025195 654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:37.025177 655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:37.025197 657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:37.025540 649 server_base.cc:1034] running on GCE node
I20260926 13:54:37.026096 649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:37.027289 649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:37.028491 649 hybrid_clock.cc:648] HybridClock initialized: now 1790430877028470 us; error 49 us; skew 500 ppm
Sep 26 13:54:37 dist-test-slave-r34j krb5kdc[636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430877, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:37.038856 649 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:37.039925 649 webserver.cc:460] Webserver started at http://127.10.236.254:37759/ using document root <none> and password file <none>
I20260926 13:54:37.040587 649 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:37.040680 649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:37.040902 649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:37.042999 649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "d1cfbc776d464acb9763896705011e54"
format_stamp: "Formatted at 2026-09-26 13:54:37 on dist-test-slave-r34j"
server_key: "aa4a6e262d7b4de4417615f6a9794777"
server_key_iv: "ce0748aeb4e51d31345fcdbf6cf38d8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:37.043637 649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "d1cfbc776d464acb9763896705011e54"
format_stamp: "Formatted at 2026-09-26 13:54:37 on dist-test-slave-r34j"
server_key: "aa4a6e262d7b4de4417615f6a9794777"
server_key_iv: "ce0748aeb4e51d31345fcdbf6cf38d8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:37.047319 649 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:37.049780 664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:37.051167 649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:54:37.051328 649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "d1cfbc776d464acb9763896705011e54"
format_stamp: "Formatted at 2026-09-26 13:54:37 on dist-test-slave-r34j"
server_key: "aa4a6e262d7b4de4417615f6a9794777"
server_key_iv: "ce0748aeb4e51d31345fcdbf6cf38d8a"
server_key_version: "encryptionkey@0"
I20260926 13:54:37.051450 649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:37.071013 649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:37.074564 649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:37.074793 649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:37.082233 649 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:42751
I20260926 13:54:37.082261 716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:42751 every 8 connection(s)
I20260926 13:54:37.083189 649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:37.085942 717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:37.088836 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 649
I20260926 13:54:37.088999 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:37.089282 11187 external_mini_cluster.cc:1428] Setting key 8060440c075167ce6b5c3fdc83536d5d
I20260926 13:54:37.092185 717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54: Bootstrap starting.
I20260926 13:54:37.094332 717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:37.095065 717 log.cc:824] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:37.097077 717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54: No bootstrap required, opened a new log
Sep 26 13:54:37 dist-test-slave-r34j krb5kdc[636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:37.100386 717 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1cfbc776d464acb9763896705011e54" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42751 } }
I20260926 13:54:37.100660 717 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:37.100744 717 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1cfbc776d464acb9763896705011e54, State: Initialized, Role: FOLLOWER
I20260926 13:54:37.101191 717 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1cfbc776d464acb9763896705011e54" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42751 } }
I20260926 13:54:37.101339 717 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:37.101425 717 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:37.101533 717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:37.102418 717 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1cfbc776d464acb9763896705011e54" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42751 } }
I20260926 13:54:37.102757 717 leader_election.cc:302] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1cfbc776d464acb9763896705011e54; no voters:
I20260926 13:54:37.103120 717 leader_election.cc:288] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:37.103209 722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:37.103422 722 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [term 1 LEADER]: Becoming Leader. State: Replica: d1cfbc776d464acb9763896705011e54, State: Running, Role: LEADER
I20260926 13:54:37.103811 722 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1cfbc776d464acb9763896705011e54" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42751 } }
I20260926 13:54:37.104437 720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:37.090801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49241 (local address 127.10.236.254:42751)
0926 13:54:37.091270 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:37.091285 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:37.091320 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:54:37.091986 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:37.092003 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:37.092392 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:37.092760 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:37.093531 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:37.094556 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:37.095470 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:37.095968 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:37.096131 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:37.098883 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:37.098888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:37.098901 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:37.098927 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:37.101319 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:37.101847 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:37.101853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:37.101858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:37.101975 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:37.102272 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:37.102276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:37.102278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:37.102739 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:37.102929 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:37.103232 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 13:54:37.103595 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":143,"threads_started":1}
I20260926 13:54:37.104493 717 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:37.105613 723 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1cfbc776d464acb9763896705011e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1cfbc776d464acb9763896705011e54" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42751 } } }
I20260926 13:54:37.105736 723 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:37.106006 724 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1cfbc776d464acb9763896705011e54. Latest consensus state: current_term: 1 leader_uuid: "d1cfbc776d464acb9763896705011e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1cfbc776d464acb9763896705011e54" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 42751 } } }
I20260926 13:54:37.106119 731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:37.106153 724 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:37.107705 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:37.109506 731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:37.116191 731 catalog_manager.cc:1347] Generated new cluster ID: 2ef33a20e82c4aec8a2412bdc0746412
I20260926 13:54:37.116328 731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:37.119899 720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:37.110260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48595 (local address 127.10.236.254:42751)
0926 13:54:37.110380 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:37.110384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:37.110622 (+ 238us) server_negotiation.cc:407] Connection header received
0926 13:54:37.110821 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:37.110824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:37.110871 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:37.110953 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:37.111845 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:37.112624 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:37.113816 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:37.114048 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:37.114185 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:37.115870 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:37.115873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:37.115876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:37.115901 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:37.117690 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:37.118206 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:37.118211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:37.118215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:37.118302 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:37.118607 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:37.118610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:37.118612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:37.118828 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:37.118948 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:37.119606 (+ 658us) server_negotiation.cc:299] Negotiation successful
0926 13:54:37.119728 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:54:37.158921 731 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:37.160051 731 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:37.167979 731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1cfbc776d464acb9763896705011e54: Generated new TSK 0
I20260926 13:54:37.168704 731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:37.174888 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 649
2026-09-26T13:54:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:37 dist-test-slave-r34j krb5kdc[748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:37 dist-test-slave-r34j krb5kdc[748](info): set up 2 sockets
Sep 26 13:54:37 dist-test-slave-r34j krb5kdc[748](info): commencing operation
krb5kdc: starting...
W20260926 13:54:38.303884 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:38 dist-test-slave-r34j krb5kdc[748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:38.435737 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44817
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:38.547351 761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:38.547760 761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:38.547842 761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:38.553118 761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:38.553236 761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:38.553277 761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:38.553315 761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:38.553349 761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:38.559186 761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:38.560387 761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:38.561475 761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:38.570454 766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:38.570487 769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:38.570497 767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:38.570544 761 server_base.cc:1034] running on GCE node
I20260926 13:54:38.571359 761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:38.572563 761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:38.573752 761 hybrid_clock.cc:648] HybridClock initialized: now 1790430878573740 us; error 31 us; skew 500 ppm
Sep 26 13:54:38 dist-test-slave-r34j krb5kdc[748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430878, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:38.576849 761 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:38.577870 761 webserver.cc:460] Webserver started at http://127.10.236.254:33797/ using document root <none> and password file <none>
I20260926 13:54:38.578485 761 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:38.578549 761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:38.578740 761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:38.581550 761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "88e57c5258bf427fb0e488ae39e427ea"
format_stamp: "Formatted at 2026-09-26 13:54:38 on dist-test-slave-r34j"
server_key: "10d2cc8e4020e37ed658133e471292c6"
server_key_iv: "26d61be97d3ab4ecad9a58bc4680e8af"
server_key_version: "encryptionkey@0"
I20260926 13:54:38.582350 761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "88e57c5258bf427fb0e488ae39e427ea"
format_stamp: "Formatted at 2026-09-26 13:54:38 on dist-test-slave-r34j"
server_key: "10d2cc8e4020e37ed658133e471292c6"
server_key_iv: "26d61be97d3ab4ecad9a58bc4680e8af"
server_key_version: "encryptionkey@0"
I20260926 13:54:38.587088 761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:38.590672 776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:38.592212 761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:38.592343 761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "88e57c5258bf427fb0e488ae39e427ea"
format_stamp: "Formatted at 2026-09-26 13:54:38 on dist-test-slave-r34j"
server_key: "10d2cc8e4020e37ed658133e471292c6"
server_key_iv: "26d61be97d3ab4ecad9a58bc4680e8af"
server_key_version: "encryptionkey@0"
I20260926 13:54:38.592468 761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:38.605695 761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:38.609083 761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:38.609274 761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:38.616783 761 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44817
I20260926 13:54:38.616864 828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44817 every 8 connection(s)
I20260926 13:54:38.617803 761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:38.620898 829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:38.622505 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 761
I20260926 13:54:38.622663 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:38.622934 11187 external_mini_cluster.cc:1428] Setting key 3af8e6a46a0ac954fc7239146d38b8ec
I20260926 13:54:38.628204 829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea: Bootstrap starting.
I20260926 13:54:38.631031 829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:38.631980 829 log.cc:824] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:38.633955 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:38.624464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49313 (local address 127.10.236.254:44817)
0926 13:54:38.624928 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:38.624949 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:38.624996 (+ 47us) server_negotiation.cc:407] Connection header received
0926 13:54:38.625924 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:38.625949 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:38.626433 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:38.626858 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:38.627322 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:38.628946 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:38.630066 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:38.630835 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:38.631034 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:38.632106 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:38.632113 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:38.632130 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:38.632635 (+ 505us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:38.632690 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:38.632713 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:38.632838 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:38.632981 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 13:54:38.633205 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260926 13:54:38.634339 829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea: No bootstrap required, opened a new log
I20260926 13:54:38.637295 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:38.639052 829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e57c5258bf427fb0e488ae39e427ea" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44817 } }
I20260926 13:54:38.639374 829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:38.639451 829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e57c5258bf427fb0e488ae39e427ea, State: Initialized, Role: FOLLOWER
I20260926 13:54:38.640031 829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e57c5258bf427fb0e488ae39e427ea" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44817 } }
I20260926 13:54:38.640208 829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:38.640288 829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:38.640427 829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:38.641520 829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e57c5258bf427fb0e488ae39e427ea" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44817 } }
I20260926 13:54:38.641913 829 leader_election.cc:302] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e57c5258bf427fb0e488ae39e427ea; no voters:
I20260926 13:54:38.642294 829 leader_election.cc:288] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:38.642419 839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:38.642669 839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [term 1 LEADER]: Becoming Leader. State: Replica: 88e57c5258bf427fb0e488ae39e427ea, State: Running, Role: LEADER
I20260926 13:54:38.643028 839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e57c5258bf427fb0e488ae39e427ea" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44817 } }
I20260926 13:54:38.643745 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:38.639438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43527 (local address 127.10.236.254:44817)
0926 13:54:38.639589 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:38.639596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:38.639881 (+ 285us) server_negotiation.cc:407] Connection header received
0926 13:54:38.640037 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:38.640039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:38.640091 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:38.640177 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:38.640441 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:38.641015 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:38.641912 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:38.642225 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:38.642337 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:38.643315 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:38.643318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:38.643320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:38.643331 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:38.643355 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:38.643365 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:38.643445 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:38.643558 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 13:54:38.643610 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:54:38.644239 829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:38.644913 841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e57c5258bf427fb0e488ae39e427ea. Latest consensus state: current_term: 1 leader_uuid: "88e57c5258bf427fb0e488ae39e427ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e57c5258bf427fb0e488ae39e427ea" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44817 } } }
I20260926 13:54:38.645051 841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:38.645439 845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:38.645390 840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e57c5258bf427fb0e488ae39e427ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e57c5258bf427fb0e488ae39e427ea" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44817 } } }
I20260926 13:54:38.645556 840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:38.648866 845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:38.655098 845 catalog_manager.cc:1347] Generated new cluster ID: 23de7401acae4c61b1928b107401b912
I20260926 13:54:38.655176 845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:38.673501 845 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:38.674940 845 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:38.693409 845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e57c5258bf427fb0e488ae39e427ea: Generated new TSK 0
I20260926 13:54:38.694191 845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:38.706843 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 761
2026-09-26T13:54:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:38 dist-test-slave-r34j krb5kdc[860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:38 dist-test-slave-r34j krb5kdc[860](info): set up 2 sockets
Sep 26 13:54:38 dist-test-slave-r34j krb5kdc[860](info): commencing operation
krb5kdc: starting...
W20260926 13:54:39.776036 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:39 dist-test-slave-r34j krb5kdc[860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:39.908017 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44591
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:44591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:40.025854 873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:40.026161 873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:40.026206 873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:40.030416 873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:40.030493 873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:40.030514 873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:40.030531 873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:40.030546 873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:40.036644 873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:44591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:44591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:40.037928 873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:40.039012 873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:40.047816 879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:40.047837 881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:40.047821 878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:40.048168 873 server_base.cc:1034] running on GCE node
I20260926 13:54:40.048812 873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:40.050191 873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:40.051550 873 hybrid_clock.cc:648] HybridClock initialized: now 1790430880051515 us; error 49 us; skew 500 ppm
Sep 26 13:54:40 dist-test-slave-r34j krb5kdc[860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430880, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:40.054566 873 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:40.055457 873 webserver.cc:460] Webserver started at http://127.10.236.254:44985/ using document root <none> and password file <none>
I20260926 13:54:40.056155 873 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:40.056218 873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:40.056387 873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:40.058421 873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "a29b262fa4cc4599b9c117ecb9f8a73e"
format_stamp: "Formatted at 2026-09-26 13:54:40 on dist-test-slave-r34j"
server_key: "6a2e42d0b6a7f65ff8886729286ef764"
server_key_iv: "0c020d712bfaf9eef19b709c8b259850"
server_key_version: "encryptionkey@0"
I20260926 13:54:40.058966 873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "a29b262fa4cc4599b9c117ecb9f8a73e"
format_stamp: "Formatted at 2026-09-26 13:54:40 on dist-test-slave-r34j"
server_key: "6a2e42d0b6a7f65ff8886729286ef764"
server_key_iv: "0c020d712bfaf9eef19b709c8b259850"
server_key_version: "encryptionkey@0"
I20260926 13:54:40.063184 873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:40.066159 888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:40.067907 873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 13:54:40.068049 873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "a29b262fa4cc4599b9c117ecb9f8a73e"
format_stamp: "Formatted at 2026-09-26 13:54:40 on dist-test-slave-r34j"
server_key: "6a2e42d0b6a7f65ff8886729286ef764"
server_key_iv: "0c020d712bfaf9eef19b709c8b259850"
server_key_version: "encryptionkey@0"
I20260926 13:54:40.068167 873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:40.073534 873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:40.076884 873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:40.077052 873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:40.084250 873 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:44591
I20260926 13:54:40.084362 940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:44591 every 8 connection(s)
I20260926 13:54:40.085281 873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:40.088255 941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:40.093947 941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e: Bootstrap starting.
I20260926 13:54:40.095134 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 873
I20260926 13:54:40.095291 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:40.095638 11187 external_mini_cluster.cc:1428] Setting key 400468fa9c8ddc75d2a24d030244dd4e
I20260926 13:54:40.096715 941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:40.097548 941 log.cc:824] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:40.099610 941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e: No bootstrap required, opened a new log
I20260926 13:54:40.100800 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:40.096854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47155 (local address 127.10.236.254:44591)
0926 13:54:40.097333 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:40.097349 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:40.097385 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:54:40.098092 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:40.098110 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:40.098385 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:40.098733 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:40.099531 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:40.099536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:40.099557 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:40.099568 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:40.099619 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:40.099639 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:40.099797 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:40.099909 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 13:54:40.100073 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":134,"threads_started":1}
I20260926 13:54:40.103574 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:40.103389 941 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44591 } }
I20260926 13:54:40.103673 941 raft_consensus.cc:374] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:54:40.103761 941 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29b262fa4cc4599b9c117ecb9f8a73e, State: Initialized, Role: FOLLOWER
I20260926 13:54:40.104238 941 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44591 } }
I20260926 13:54:40.104362 941 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:54:40.104444 941 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:54:40.104547 941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:40.105711 941 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44591 } }
I20260926 13:54:40.106094 941 leader_election.cc:302] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29b262fa4cc4599b9c117ecb9f8a73e; no voters:
I20260926 13:54:40.106474 941 leader_election.cc:288] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:54:40.106580 951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:40.106806 951 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [term 1 LEADER]: Becoming Leader. State: Replica: a29b262fa4cc4599b9c117ecb9f8a73e, State: Running, Role: LEADER
I20260926 13:54:40.107187 951 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44591 } }
I20260926 13:54:40.107687 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:40.105857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59963 (local address 127.10.236.254:44591)
0926 13:54:40.105965 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:40.105969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:40.106185 (+ 216us) server_negotiation.cc:407] Connection header received
0926 13:54:40.106292 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:40.106295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:40.106342 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:40.106409 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:40.107172 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:40.107176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:40.107178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:54:40.107190 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:40.107219 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:54:40.107232 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:40.107320 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:40.107538 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 13:54:40.107579 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 13:54:40.107918 941 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:40.109118 952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29b262fa4cc4599b9c117ecb9f8a73e. Latest consensus state: current_term: 1 leader_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44591 } } }
I20260926 13:54:40.109118 953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29b262fa4cc4599b9c117ecb9f8a73e" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 44591 } } }
I20260926 13:54:40.109284 952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:40.109290 953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:40.109678 960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:40.112692 960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:40.119184 960 catalog_manager.cc:1347] Generated new cluster ID: 315c6f0b850b4277a8a7d76954fb48bb
I20260926 13:54:40.119272 960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:40.127972 960 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:40.128340 960 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:40.133569 960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29b262fa4cc4599b9c117ecb9f8a73e: Generated new TSK 0
I20260926 13:54:40.134300 960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:54:40.141713 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 873
2026-09-26T13:54:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1434 ms)
[----------] 18 tests from AuthTokenIssuingTest (26259 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:40 dist-test-slave-r34j krb5kdc[972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:40 dist-test-slave-r34j krb5kdc[972](info): set up 2 sockets
Sep 26 13:54:40 dist-test-slave-r34j krb5kdc[972](info): commencing operation
krb5kdc: starting...
W20260926 13:54:41.208112 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:41.334897 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37411
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:37411,127.10.236.253:41405,127.10.236.252:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:41.448794 985 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:54:41.449162 985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:41.449241 985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:41.449328 985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:41.453419 985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:41.453526 985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:41.453564 985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:41.453604 985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:41.453639 985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:41.458995 985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:37411,127.10.236.253:41405,127.10.236.252:42769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:37411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:41.460163 985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:41.461411 985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:41.470645 991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:41.470845 985 server_base.cc:1034] running on GCE node
W20260926 13:54:41.470675 990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:41.470667 993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:41.471666 985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:41.472800 985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:41.473999 985 hybrid_clock.cc:648] HybridClock initialized: now 1790430881473965 us; error 47 us; skew 500 ppm
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:41.476889 985 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:41.477955 985 webserver.cc:460] Webserver started at http://127.10.236.254:39847/ using document root <none> and password file <none>
I20260926 13:54:41.478546 985 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:41.478612 985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:41.478799 985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:41.481324 985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "ca8b6e015b9843cda87a2172e1064a90"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "fd11440d876124465a5e38cb89a9863d"
server_key_iv: "57eac2474074205ed02d83bb9bf2d7c8"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.481936 985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "ca8b6e015b9843cda87a2172e1064a90"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "fd11440d876124465a5e38cb89a9863d"
server_key_iv: "57eac2474074205ed02d83bb9bf2d7c8"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.486195 985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 13:54:41.489056 1000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:41.490422 985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:54:41.490593 985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "ca8b6e015b9843cda87a2172e1064a90"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "fd11440d876124465a5e38cb89a9863d"
server_key_iv: "57eac2474074205ed02d83bb9bf2d7c8"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.490746 985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:41.531193 985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:41.534736 985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:41.534972 985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:41.542695 985 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:37411
I20260926 13:54:41.542984 1052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:37411 every 8 connection(s)
I20260926 13:54:41.543855 985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:41.546620 1053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:41.550494 1053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:41.552433 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 985
I20260926 13:54:41.552587 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:41.552917 11187 external_mini_cluster.cc:1428] Setting key d73b6e27ad4b0e6c707412e1a383ac17
WARNING: no policy specified for kudu/127.10.236.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.253@KRBTEST.COM" created.
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:41.572940 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.552923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37411 (local address 127.0.0.1:51562)
0926 13:54:41.553525 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.553580 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:41.554989 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:41.555981 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:41.555989 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:41.556531 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:41.556922 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.556944 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.558502 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.558508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.559853 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.559865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.560576 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.560582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.560711 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.561310 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:41.561333 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:41.566737 (+ 5404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:41.569487 (+ 2750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.569496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.569504 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.569756 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.570208 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.570211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.570213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.570412 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.571329 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:41.571335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:41.571468 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 13:54:41.571700 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 13:54:41.571990 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"spinlock_wait_cycles":58624,"thread_start_us":191,"threads_started":1}
I20260926 13:54:41.572944 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.552924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.10.236.254:37411)
0926 13:54:41.553623 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:41.553638 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:41.553916 (+ 278us) server_negotiation.cc:407] Connection header received
0926 13:54:41.555454 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:41.555459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:41.555759 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:41.555934 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:41.557144 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.558355 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.559993 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.560393 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.560530 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.566962 (+ 6432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:41.566974 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:41.566994 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:41.567033 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:41.569335 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.569884 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.569895 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.569900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.570066 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.570551 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.570558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.570563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.571183 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:41.571371 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:41.571825 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 13:54:41.572123 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":555,"thread_start_us":200,"threads_started":1}
I20260926 13:54:41.575799 1053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:41.576709 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.576309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45376)
0926 13:54:41.576559 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.576640 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260926 13:54:41.576973 1003 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
I20260926 13:54:41.578047 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.577708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45382)
0926 13:54:41.577901 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.577965 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260926 13:54:41.578227 1053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:41405: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab.
Entry for principal kudu/127.10.236.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab.
I20260926 13:54:41.587100 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:41405
--webserver_interface=127.10.236.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab
--principal=kudu/127.10.236.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:37411,127.10.236.253:41405,127.10.236.252:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:54:41.627543 1053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } attempt: 1
I20260926 13:54:41.628588 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.628284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45394)
0926 13:54:41.628449 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.628512 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 13:54:41.629585 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.629333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45400)
0926 13:54:41.629477 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.629525 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 13:54:41.629766 1053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:41405: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
I20260926 13:54:41.697945 1053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } attempt: 2
I20260926 13:54:41.698833 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.698497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45414)
0926 13:54:41.698662 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.698751 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 13:54:41.699755 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.699452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45420)
0926 13:54:41.699611 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.699690 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260926 13:54:41.699949 1053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:41405: Network error: Client connection negotiation failed: client connection to 127.10.236.253:41405: connect: Connection refused (error 111)
W20260926 13:54:41.702221 1060 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:54:41.702498 1060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:41.702522 1060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:41.702601 1060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:41.706818 1060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:41.706894 1060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:41.706915 1060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:41.706933 1060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:41.706988 1060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:41.712440 1060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:37411,127.10.236.253:41405,127.10.236.252:42769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab
--principal=kudu/127.10.236.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.236.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:41.713531 1060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:41.714524 1060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:41.723101 1066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:41.723122 1068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:41.723101 1065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:41.723371 1060 server_base.cc:1034] running on GCE node
I20260926 13:54:41.724020 1060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:41.725171 1060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:41.726377 1060 hybrid_clock.cc:648] HybridClock initialized: now 1790430881726363 us; error 35 us; skew 500 ppm
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:41.729389 1060 init.cc:377] Logged in from keytab as kudu/127.10.236.253@KRBTEST.COM (short username kudu)
I20260926 13:54:41.730309 1060 webserver.cc:460] Webserver started at http://127.10.236.253:36233/ using document root <none> and password file <none>
I20260926 13:54:41.730875 1060 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:41.730966 1060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:41.731211 1060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:41.733330 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data/instance:
uuid: "245c4028ab0f4f44af9a8bfbe3a37c13"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "5f9c8e6b8ec316f422c5870f4ac760f7"
server_key_iv: "fd562a4034a097ad896149fa2837a93f"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.733898 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal/instance:
uuid: "245c4028ab0f4f44af9a8bfbe3a37c13"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "5f9c8e6b8ec316f422c5870f4ac760f7"
server_key_iv: "fd562a4034a097ad896149fa2837a93f"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.737673 1060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 13:54:41.740151 1075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:41.741381 1060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:54:41.741528 1060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal
uuid: "245c4028ab0f4f44af9a8bfbe3a37c13"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "5f9c8e6b8ec316f422c5870f4ac760f7"
server_key_iv: "fd562a4034a097ad896149fa2837a93f"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.741647 1060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:41.756107 1060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:41.759220 1060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:41.759426 1060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:41.766940 1060 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.253:41405
I20260926 13:54:41.767238 1127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.253:41405 every 8 connection(s)
I20260926 13:54:41.767966 1060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
I20260926 13:54:41.770977 1128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:41.773924 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 1060
I20260926 13:54:41.774124 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-1/wal/instance
I20260926 13:54:41.774523 11187 external_mini_cluster.cc:1428] Setting key 75b6a441a4e93cde08efad2560ed4add
I20260926 13:54:41.775354 1128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.236.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.252@KRBTEST.COM" created.
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:41.791023 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.777537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.10.236.254:37411)
0926 13:54:41.777713 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:41.777718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:41.778713 (+ 995us) server_negotiation.cc:407] Connection header received
0926 13:54:41.779833 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:41.779839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:41.779912 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:41.780034 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:41.781262 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.782158 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.783431 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.783958 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.784049 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.786968 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:41.786972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:41.786974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:41.787000 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:41.788504 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.789173 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.789176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.789177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.789272 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.789701 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.789704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.789705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.789911 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:41.789997 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:41.790772 (+ 775us) server_negotiation.cc:299] Negotiation successful
0926 13:54:41.790885 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 13:54:41.791123 1053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } attempt: 3
I20260926 13:54:41.792623 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.777981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37411 (local address 127.0.0.1:51578)
0926 13:54:41.778504 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.778561 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:41.779602 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:41.780218 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:41.780225 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:41.780586 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:41.781089 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.781104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.782304 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.782308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.783296 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.783306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.784125 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.784128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.784296 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.784964 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:41.784987 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:41.786762 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:41.788646 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.788656 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.788676 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.789050 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.789378 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.789384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.789389 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.789581 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.790033 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:41.790042 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:41.790442 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 13:54:41.790676 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 13:54:41.791073 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":162,"threads_started":1}
I20260926 13:54:41.794162 1128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab.
Entry for principal kudu/127.10.236.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab.
I20260926 13:54:41.802878 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.791622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45434)
0926 13:54:41.791762 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.791778 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:41.791854 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:41.792485 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:41.792488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:41.792663 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:41.792824 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.792834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.794219 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.794225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.795466 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.795475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.796249 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.796254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.796365 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.796884 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:41.796898 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:41.798230 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:41.800783 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.800787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.800789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.801047 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.801386 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.801388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.801390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.801512 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.802336 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:41.802344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:41.802431 (+ 87us) client_negotiation.cc:769] Sending connection context
0926 13:54:41.802546 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 13:54:41.802694 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 13:54:41.803020 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.791831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.10.236.253:41405)
0926 13:54:41.792131 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:41.792141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:41.792159 (+ 18us) server_negotiation.cc:407] Connection header received
0926 13:54:41.792196 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:41.792200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:41.792356 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:41.792504 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:41.792951 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.794102 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.795611 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.796117 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.796297 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.798362 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:41.798372 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:41.798388 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:41.798414 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:41.800666 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.801187 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.801194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.801196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.801293 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.801633 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.801636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.801638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.801929 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:41.802101 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:41.802652 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 13:54:41.802803 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":196,"thread_start_us":113,"threads_started":1}
I20260926 13:54:41.803321 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:42769
--webserver_interface=127.10.236.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab
--principal=kudu/127.10.236.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:37411,127.10.236.253:41405,127.10.236.252:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:54:41.805296 1053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:41.806114 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.805798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50710)
0926 13:54:41.805986 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.806040 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 13:54:41.806191 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.794753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45442)
0926 13:54:41.794882 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.794914 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:41.794993 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:41.796220 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:41.796223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:41.796498 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:41.796578 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.796583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.798011 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.798015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.798756 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:41.798772 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.799331 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.799335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:41.799455 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.799907 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:41.799923 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:41.801622 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:41.804525 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.804528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.804530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.804762 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.805040 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:41.805045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:41.805048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:41.805153 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:41.805614 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:41.805618 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:41.805704 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 13:54:41.805823 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 13:54:41.805986 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 13:54:41.806242 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.795666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.10.236.253:41405)
0926 13:54:41.796018 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:41.796034 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:41.796046 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:54:41.796074 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:41.796076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:41.796108 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:41.796214 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:41.796692 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.797860 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.798953 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:41.799212 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:41.799297 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:41.801751 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:41.801753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:41.801755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:41.801795 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:41.804417 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.804876 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.804878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.804880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.804954 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:41.805263 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:41.805267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:41.805270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:41.805488 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:41.805578 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:41.805884 (+ 306us) server_negotiation.cc:299] Negotiation successful
0926 13:54:41.806055 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":279,"thread_start_us":135,"threads_started":1}
I20260926 13:54:41.806917 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.806628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50726)
0926 13:54:41.806780 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.806835 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260926 13:54:41.807063 1053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:42769: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
I20260926 13:54:41.807543 1128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:41.808228 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.807984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50732)
0926 13:54:41.808097 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.808175 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 13:54:41.808420 1077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
I20260926 13:54:41.809273 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.809071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50738)
0926 13:54:41.809175 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.809218 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 13:54:41.809425 1128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:42769: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
I20260926 13:54:41.838266 1053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } attempt: 1
I20260926 13:54:41.839228 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.838882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50746)
0926 13:54:41.839056 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.839129 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 13:54:41.840026 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.839771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50762)
0926 13:54:41.839909 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.839962 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 13:54:41.840220 1053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:42769: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
I20260926 13:54:41.858701 1128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } attempt: 1
I20260926 13:54:41.859989 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.859644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50764)
0926 13:54:41.859820 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.859895 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 13:54:41.860826 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.860586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50772)
0926 13:54:41.860698 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.860749 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 13:54:41.860991 1128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:42769: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
I20260926 13:54:41.915437 1053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } attempt: 2
I20260926 13:54:41.916455 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.916063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50784)
0926 13:54:41.916286 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.916373 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260926 13:54:41.917433 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.917083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50800)
0926 13:54:41.917299 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.917369 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260926 13:54:41.917598 1053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:42769: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
W20260926 13:54:41.921195 1135 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:54:41.921540 1135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:41.921605 1135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:41.921685 1135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:41.925994 1135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:41.926100 1135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:41.926152 1135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:41.926210 1135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:41.926247 1135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:41.929208 1128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } attempt: 2
I20260926 13:54:41.930159 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.929799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50806)
0926 13:54:41.929968 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.930064 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 13:54:41.930929 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:41.930710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50818)
0926 13:54:41.930815 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:41.930866 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 13:54:41.931109 1128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:42769: Network error: Client connection negotiation failed: client connection to 127.10.236.252:42769: connect: Connection refused (error 111)
I20260926 13:54:41.932675 1135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:37411,127.10.236.253:41405,127.10.236.252:42769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab
--principal=kudu/127.10.236.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.236.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:41.933862 1135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:41.934906 1135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:54:41.944444 1135 server_base.cc:1034] running on GCE node
W20260926 13:54:41.944351 1141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:41.944337 1144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:41.944314 1142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:41.945344 1135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:41.946640 1135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:41.947808 1135 hybrid_clock.cc:648] HybridClock initialized: now 1790430881947788 us; error 35 us; skew 500 ppm
Sep 26 13:54:41 dist-test-slave-r34j krb5kdc[972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:41.950831 1135 init.cc:377] Logged in from keytab as kudu/127.10.236.252@KRBTEST.COM (short username kudu)
I20260926 13:54:41.951859 1135 webserver.cc:460] Webserver started at http://127.10.236.252:41095/ using document root <none> and password file <none>
I20260926 13:54:41.952435 1135 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:41.952497 1135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:41.952729 1135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:41.954815 1135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data/instance:
uuid: "071d24caa48446e8ae56a4993649985c"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "e560cc9283e3ca42a7116cc298e152e7"
server_key_iv: "fdfb00104fb388b620ca7544e8e20b35"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.955407 1135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal/instance:
uuid: "071d24caa48446e8ae56a4993649985c"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "e560cc9283e3ca42a7116cc298e152e7"
server_key_iv: "fdfb00104fb388b620ca7544e8e20b35"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.959326 1135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 13:54:41.962288 1151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:41.964160 1135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:41.964286 1135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal
uuid: "071d24caa48446e8ae56a4993649985c"
format_stamp: "Formatted at 2026-09-26 13:54:41 on dist-test-slave-r34j"
server_key: "e560cc9283e3ca42a7116cc298e152e7"
server_key_iv: "fdfb00104fb388b620ca7544e8e20b35"
server_key_version: "encryptionkey@0"
I20260926 13:54:41.964412 1135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:41.979447 1135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:41.983155 1135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:41.983364 1135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:41.992034 1135 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.252:42769
I20260926 13:54:41.992395 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.252:42769 every 8 connection(s)
I20260926 13:54:41.993253 1135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
I20260926 13:54:41.996553 1204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:42.000587 1204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:42.001533 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 1135
I20260926 13:54:42.001679 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790430594225530-11187-0/minicluster-data/master-2/wal/instance
I20260926 13:54:42.001940 11187 external_mini_cluster.cc:1428] Setting key cf4ae6b8a9c9e0688d3b46e8b2cb78cd
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:54:42.014688 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.002693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.10.236.254:37411)
0926 13:54:42.002924 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.002928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.003980 (+ 1052us) server_negotiation.cc:407] Connection header received
0926 13:54:42.005004 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.005007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.005072 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.005169 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.006040 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.006554 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.007588 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.007817 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.007934 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.010863 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.010866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.010868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.010893 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.012574 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.013106 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.013110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.013112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.013198 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.013563 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.013566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.013568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.013775 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.013893 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.014393 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.014520 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 13:54:42.015661 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.003201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:37411 (local address 127.0.0.1:51590)
0926 13:54:42.003822 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:42.003879 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:42.004800 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:42.005221 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:42.005226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:42.005582 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:42.005895 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.005908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.006668 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.006673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.007439 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.007448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.007974 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.007977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.008234 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.008916 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:42.008939 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:42.010678 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:42.012696 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.012706 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.012717 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.012997 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.013308 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.013311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.013313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.013458 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.013893 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:42.013901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:42.014165 (+ 264us) client_negotiation.cc:769] Sending connection context
0926 13:54:42.014354 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 13:54:42.014695 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":52,"thread_start_us":198,"threads_started":1}
I20260926 13:54:42.015663 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.004140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.10.236.252:42769)
0926 13:54:42.004409 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.004418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.004450 (+ 32us) server_negotiation.cc:407] Connection header received
0926 13:54:42.004810 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.004831 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.005040 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.005302 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.006101 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.006959 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.007888 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.008363 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.008481 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.011207 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.011214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.011224 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.011245 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.012970 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.013427 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.013431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.013432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.013515 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.013804 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.013810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.013813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.014155 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.014384 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.014562 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.014767 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":172,"spinlock_wait_cycles":8960,"thread_start_us":105,"threads_started":1}
I20260926 13:54:42.016796 1053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } attempt: 3
I20260926 13:54:42.017256 1204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:42.022357 1128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } attempt: 3
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:54:42.028198 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.017190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50842)
0926 13:54:42.017359 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:42.017373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:42.017449 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:42.017724 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:42.017727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:42.017927 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:42.018027 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.018036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.018998 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.019004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.019862 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.019869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.021238 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.021243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.021338 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.021855 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:42.021872 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:42.023919 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:42.026558 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.026561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.026563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.026801 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.027114 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.027117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.027119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.027201 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.027653 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:42.027656 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:42.027754 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 13:54:42.027841 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 13:54:42.027979 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":18}
I20260926 13:54:42.028504 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.018001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.10.236.254:37411)
0926 13:54:42.018111 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.018115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.018155 (+ 40us) server_negotiation.cc:407] Connection header received
0926 13:54:42.018280 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.018285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.018332 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.018433 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.019055 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.019854 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.020890 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.021217 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.021311 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.024440 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.024444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.024446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.024473 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.026270 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.026810 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.026815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.026818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.026937 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.027293 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.027297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.027298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.027579 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.027806 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.028198 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.028327 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":37}
I20260926 13:54:42.028754 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.017277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.10.236.252:42769)
0926 13:54:42.017395 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.017398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.017412 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:54:42.017565 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.017568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.017612 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.017707 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.018337 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.018885 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.020889 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.021109 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.021224 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.024604 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.024610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.024613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.024641 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.026439 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.026940 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.026943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.026945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.027020 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.027324 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.027327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.027328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.027543 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.028399 (+ 856us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.028469 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.028579 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:54:42.029871 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.018265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45450 (local address 127.10.236.253:41405)
0926 13:54:42.018427 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.018432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.019085 (+ 653us) server_negotiation.cc:407] Connection header received
0926 13:54:42.019188 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.019191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.019241 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.019329 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.019770 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.020553 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.022033 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.022444 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.022560 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.025547 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.025554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.025798 (+ 244us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.025831 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.027617 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.028446 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.028449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.028451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.028541 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.028934 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.028937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.028939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.029137 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.029212 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.029585 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.029703 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 13:54:42.030114 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.017926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:41405 (local address 127.0.0.1:45450)
0926 13:54:42.019013 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:42.019022 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:42.019110 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:42.019343 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:42.019345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:42.019560 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:42.019654 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.019659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.020665 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.020667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.021425 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.021434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.022685 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.022689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.022841 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.023380 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:42.023391 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:42.025132 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:42.027741 (+ 2609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.027744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.027746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.027984 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.028749 (+ 765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.028752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.028754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.028837 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.029291 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:42.029296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:42.029397 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 13:54:42.029504 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 13:54:42.029966 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038}
I20260926 13:54:42.033161 1053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90: Bootstrap starting.
I20260926 13:54:42.034008 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.023364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.10.236.252:42769)
0926 13:54:42.024165 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.024173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.024184 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:54:42.024210 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.024212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.024243 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.024343 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.024813 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.026160 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.027324 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.027697 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.028123 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.030585 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.030589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.030591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.030615 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.032259 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.032717 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.032720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.032722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.032808 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.033127 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.033130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.033132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.033351 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.033437 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.033745 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.033854 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":193,"threads_started":1}
I20260926 13:54:42.033912 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.022789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50858)
0926 13:54:42.022975 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:42.022988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:42.023079 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:42.024359 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:42.024362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:42.024562 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:42.024653 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.024659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.026301 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.026308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.027186 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.027196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.027838 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.027841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.027920 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.028544 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:42.028560 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:42.030442 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:42.032391 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.032394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.032395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.032604 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.032946 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.032948 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.032950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.033032 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.033483 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:42.033485 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:42.033564 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 13:54:42.033670 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:54:42.033804 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260926 13:54:42.036087 1053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:42.036475 1204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:42.037146 1053 log.cc:824] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:42.038992 1053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90: No bootstrap required, opened a new log
I20260926 13:54:42.038978 1128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: Bootstrap starting.
I20260926 13:54:42.041733 1128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:42.042590 1128 log.cc:824] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:42.043215 1053 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.043517 1053 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8b6e015b9843cda87a2172e1064a90, State: Initialized, Role: FOLLOWER
I20260926 13:54:42.043838 1053 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.044273 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.044399 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:42.044915 1128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: No bootstrap required, opened a new log
I20260926 13:54:42.045161 1053 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:54:42.049495 1128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.049791 1128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4028ab0f4f44af9a8bfbe3a37c13, State: Initialized, Role: FOLLOWER
I20260926 13:54:42.050207 1128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.051745 1128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 13:54:42.052700 1223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:42.052832 1223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:42.052994 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.036933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:42769 (local address 127.0.0.1:50872)
0926 13:54:42.037324 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:42.037345 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:42.037403 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:42.039422 (+ 2019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:42.039425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:42.039620 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:42.039708 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.039714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.040654 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.040657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.041441 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:42.041451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.042088 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.042092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:42.042167 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.047439 (+ 5272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:42.047457 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:42.048985 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:42.051167 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.051170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.051172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.051426 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.051781 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:42.051784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:42.051786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:42.051877 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:42.052376 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:42.052379 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:42.052468 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 13:54:42.052578 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 13:54:42.052758 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326}
I20260926 13:54:42.053695 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.037119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50872 (local address 127.10.236.252:42769)
0926 13:54:42.039206 (+ 2087us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.039211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.039224 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:54:42.039258 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.039260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.039299 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.039381 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.039842 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.040537 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.041562 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.041896 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.041990 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.049129 (+ 7139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.049132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.049135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.049158 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.051021 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.051570 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.051574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.051576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.051673 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.051979 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.051982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.051984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.052233 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.052338 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.053347 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.053476 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2020}
I20260926 13:54:42.050521 985 master_runner.cc:186] Error getting master registration for 127.10.236.253:41405: OK, instance_id { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" instance_seqno: 1790430881764786 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:54:42.053953 985 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 13:54:42 dist-test-slave-r34j krb5kdc[972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:54:42.056933 1204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: Bootstrap starting.
W20260926 13:54:42.057694 1236 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:42.057770 1236 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:42.059768 1204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:42.060411 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.030496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.10.236.253:41405)
0926 13:54:42.035626 (+ 5130us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.035629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.035641 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:54:42.035669 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.035671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.035704 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.035806 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.036703 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.037475 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.040485 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.040830 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.040905 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.056012 (+ 15107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.056017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.056020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.056048 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.058010 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.058627 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.058631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.058634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.058731 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.059166 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.059173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.059176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.059433 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.059666 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.060070 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.060207 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":5048}
I20260926 13:54:42.060643 1204 log.cc:824] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:42.059417 1060 master_runner.cc:186] Error getting master registration for 127.10.236.252:42769: OK, instance_id { permanent_uuid: "071d24caa48446e8ae56a4993649985c" instance_seqno: 1790430881989147 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:54:42.063020 1060 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:54:42.063261 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.063396 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:42.064158 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:42.065518 1204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: No bootstrap required, opened a new log
I20260926 13:54:42.065510 1027 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ca8b6e015b9843cda87a2172e1064a90"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51604
I20260926 13:54:42.065841 1027 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 13:54:42.065968 1027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:42.066992 1027 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.068041 1027 leader_election.cc:288] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [CANDIDATE]: Term 1 election: Requested vote from peers 245c4028ab0f4f44af9a8bfbe3a37c13 (127.10.236.253:41405), 071d24caa48446e8ae56a4993649985c (127.10.236.252:42769)
I20260926 13:54:42.068506 1102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca8b6e015b9843cda87a2172e1064a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13"
I20260926 13:54:42.068809 1102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:42.069933 1204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.070197 1204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071d24caa48446e8ae56a4993649985c, State: Initialized, Role: FOLLOWER
I20260926 13:54:42.070526 1204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.070628 1102 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8b6e015b9843cda87a2172e1064a90 in term 1.
I20260926 13:54:42.071688 1003 leader_election.cc:302] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4028ab0f4f44af9a8bfbe3a37c13, ca8b6e015b9843cda87a2172e1064a90; no voters:
I20260926 13:54:42.071952 1204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:42.071988 1212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:42.072191 1212 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [term 1 LEADER]: Becoming Leader. State: Replica: ca8b6e015b9843cda87a2172e1064a90, State: Running, Role: LEADER
I20260926 13:54:42.072511 1212 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } }
I20260926 13:54:42.074354 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca8b6e015b9843cda87a2172e1064a90. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.074455 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:42.076124 1178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca8b6e015b9843cda87a2172e1064a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "071d24caa48446e8ae56a4993649985c"
I20260926 13:54:42.076372 1178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:42.077675 1178 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8b6e015b9843cda87a2172e1064a90 in term 1.
I20260926 13:54:42.083436 1243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.083652 1243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:42.085726 1257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:42.086705 1257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:42.087563 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.074591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.10.236.253:41405)
0926 13:54:42.074719 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.074725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.077148 (+ 2423us) server_negotiation.cc:407] Connection header received
0926 13:54:42.077373 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.077377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.077430 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.077487 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.078958 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.079836 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.081014 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.081371 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.081458 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.083454 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.083459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.083463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.083514 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.085351 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.085894 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.085898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.085900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.086001 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.086353 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.086357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.086359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.086601 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.086706 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.087164 (+ 458us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.087296 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:54:42.093864 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.075393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50880 (local address 127.10.236.252:42769)
0926 13:54:42.075535 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.075541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.078782 (+ 3241us) server_negotiation.cc:407] Connection header received
0926 13:54:42.078909 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.078915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.078971 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.079049 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.083627 (+ 4578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.084426 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.086027 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.086376 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.086456 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.090010 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.090015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.090018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.090048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.091860 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.092391 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.092395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.092397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.092486 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.092823 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.092826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.092829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.093050 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.093196 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.093545 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.093686 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:54:42.094944 1178 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [term 1 FOLLOWER]: Refusing update from remote peer ca8b6e015b9843cda87a2172e1064a90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:42.095793 1244 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:42.099922 1243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca8b6e015b9843cda87a2172e1064a90. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.100039 1243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:42.101104 1102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [term 1 FOLLOWER]: Refusing update from remote peer ca8b6e015b9843cda87a2172e1064a90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:42.101701 1244 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:42.103144 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:42.076231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.10.236.254:37411)
0926 13:54:42.076352 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.076357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.085404 (+ 9047us) server_negotiation.cc:407] Connection header received
0926 13:54:42.085513 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.085516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.085575 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.085681 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.089130 (+ 3449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.089874 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.096946 (+ 7072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:42.097290 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:42.097370 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:42.099312 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:42.099317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:42.099320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:42.099350 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:42.101157 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.101699 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.101704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.101706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.101795 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:42.102110 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:42.102113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:42.102115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:42.102340 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:42.102447 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:42.102843 (+ 396us) server_negotiation.cc:299] Negotiation successful
0926 13:54:42.102969 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:54:42.105796 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca8b6e015b9843cda87a2172e1064a90. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.105918 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 13:54:42.106951 1255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:42.107075 1255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:42.108402 1244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.108533 1244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:42.109592 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.109756 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:42.112365 1265 mvcc.cc:205] Tried to move back new op lower bound from 7333604893042663424 to 7333604892972466176. Current Snapshot: MvccSnapshot[applied={T|T < 7333604893042663424}]
I20260926 13:54:42.112476 1261 mvcc.cc:205] Tried to move back new op lower bound from 7333604893042663424 to 7333604892972466176. Current Snapshot: MvccSnapshot[applied={T|T < 7333604893042663424}]
I20260926 13:54:42.113251 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.113365 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90 [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:42.114436 1243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca8b6e015b9843cda87a2172e1064a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca8b6e015b9843cda87a2172e1064a90" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 37411 } } peers { permanent_uuid: "245c4028ab0f4f44af9a8bfbe3a37c13" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 41405 } } peers { permanent_uuid: "071d24caa48446e8ae56a4993649985c" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 42769 } } }
I20260926 13:54:42.114612 1243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:42.115689 1263 mvcc.cc:205] Tried to move back new op lower bound from 7333604893042663424 to 7333604892972466176. Current Snapshot: MvccSnapshot[applied={T|T < 7333604893042663424}]
I20260926 13:54:42.120085 1257 catalog_manager.cc:1347] Generated new cluster ID: 8f16d9ff020d4ddab4c21bfcf8ed6a20
I20260926 13:54:42.120144 1257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:42.148351 1257 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:42.149806 1257 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:42.168090 1257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca8b6e015b9843cda87a2172e1064a90: Generated new TSK 0
I20260926 13:54:42.169261 1257 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:54:42.205215 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.201569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.10.236.253:41405)
0926 13:54:42.201727 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.201732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.203949 (+ 2217us) server_negotiation.cc:407] Connection header received
0926 13:54:42.204172 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.204176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.204235 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.204297 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.205062 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45470: BlockingRecv error: recv got EOF from 127.0.0.1:45470 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 13:54:42.206961 1209 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.202483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.10.236.252:42769)
0926 13:54:42.202623 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.202628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.205948 (+ 3320us) server_negotiation.cc:407] Connection header received
0926 13:54:42.206141 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.206144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.206199 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.206257 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.206798 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50888: BlockingRecv error: recv got EOF from 127.0.0.1:50888 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 13:54:42.208802 1056 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.203240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.10.236.254:37411)
0926 13:54:42.203363 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.203368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.207570 (+ 4202us) server_negotiation.cc:407] Connection header received
0926 13:54:42.207994 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.207998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.208057 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.208114 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.208675 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51624: BlockingRecv error: recv got EOF from 127.0.0.1:51624 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 13:54:42.213896 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.212149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.10.236.253:41405)
0926 13:54:42.212269 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.212275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.212845 (+ 570us) server_negotiation.cc:407] Connection header received
0926 13:54:42.213080 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.213084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.213166 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.213224 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.213785 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45480: BlockingRecv error: recv got EOF from 127.0.0.1:45480 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 13:54:42.219293 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.217614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45494 (local address 127.10.236.253:41405)
0926 13:54:42.217727 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.217732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.218330 (+ 598us) server_negotiation.cc:407] Connection header received
0926 13:54:42.218537 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.218540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.218594 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.218664 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.219188 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45494: BlockingRecv error: recv got EOF from 127.0.0.1:45494 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260926 13:54:42.225432 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.223747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.10.236.253:41405)
0926 13:54:42.223859 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.223864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.224442 (+ 578us) server_negotiation.cc:407] Connection header received
0926 13:54:42.224674 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.224677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.224736 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.224793 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.225327 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45508: BlockingRecv error: recv got EOF from 127.0.0.1:45508 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 13:54:42.233839 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.231984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45510 (local address 127.10.236.253:41405)
0926 13:54:42.232145 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.232150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.232776 (+ 626us) server_negotiation.cc:407] Connection header received
0926 13:54:42.233002 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.233008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.233067 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.233127 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.233717 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45510: BlockingRecv error: recv got EOF from 127.0.0.1:45510 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260926 13:54:42.246694 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.244552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.10.236.253:41405)
0926 13:54:42.244711 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.244716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.245417 (+ 701us) server_negotiation.cc:407] Connection header received
0926 13:54:42.245705 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.245709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.245774 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.245862 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.246547 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45526: BlockingRecv error: recv got EOF from 127.0.0.1:45526 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260926 13:54:42.267772 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.265768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.10.236.253:41405)
0926 13:54:42.265899 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.265905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.266539 (+ 634us) server_negotiation.cc:407] Connection header received
0926 13:54:42.266780 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.266784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.266844 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.266924 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.267639 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45530: BlockingRecv error: recv got EOF from 127.0.0.1:45530 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 13:54:42.303947 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.302538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.10.236.253:41405)
0926 13:54:42.302736 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.302741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.302764 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:54:42.302980 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.302985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.303049 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.303162 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.303820 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45540: BlockingRecv error: recv got EOF from 127.0.0.1:45540 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 13:54:42.371361 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.370277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.10.236.253:41405)
0926 13:54:42.370437 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.370440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.370531 (+ 91us) server_negotiation.cc:407] Connection header received
0926 13:54:42.370652 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.370654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.370709 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.370793 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.371264 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45542: BlockingRecv error: recv got EOF from 127.0.0.1:45542 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 13:54:42.503018 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.501767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.10.236.253:41405)
0926 13:54:42.501924 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.501928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.502014 (+ 86us) server_negotiation.cc:407] Connection header received
0926 13:54:42.502151 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.502154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.502199 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.502277 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.502869 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45544: BlockingRecv error: recv got EOF from 127.0.0.1:45544 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 13:54:42.762813 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:42.761415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.10.236.253:41405)
0926 13:54:42.761551 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:42.761554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:42.761575 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:54:42.761722 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:42.761725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:42.761774 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:42.761856 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:42.762646 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45552: BlockingRecv error: recv got EOF from 127.0.0.1:45552 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:54:43.059144 1236 catalog_manager.cc:1259] Loaded cluster ID: 8f16d9ff020d4ddab4c21bfcf8ed6a20
I20260926 13:54:43.059289 1236 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: loading cluster ID for follower catalog manager: success
I20260926 13:54:43.061089 1236 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: acquiring CA information for follower catalog manager: success
I20260926 13:54:43.062353 1236 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 245c4028ab0f4f44af9a8bfbe3a37c13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:54:43.108525 1255 catalog_manager.cc:1259] Loaded cluster ID: 8f16d9ff020d4ddab4c21bfcf8ed6a20
I20260926 13:54:43.108618 1255 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: loading cluster ID for follower catalog manager: success
I20260926 13:54:43.110345 1255 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: acquiring CA information for follower catalog manager: success
I20260926 13:54:43.111524 1255 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 071d24caa48446e8ae56a4993649985c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:54:43.281500 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:43.277104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.10.236.253:41405)
0926 13:54:43.277521 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:43.277525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:43.277678 (+ 153us) server_negotiation.cc:407] Connection header received
0926 13:54:43.277875 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:43.277883 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:43.278018 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:43.278141 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:43.278432 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.278967 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.279947 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.280188 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.280297 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:43.280636 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:43.281083 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:43.281232 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:43.281268 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:54:43.281324 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":91,"threads_started":1}
I20260926 13:54:43.288494 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:43.284901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.10.236.252:42769)
0926 13:54:43.285134 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:43.285138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:43.285152 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:54:43.285229 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:43.285231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:43.285342 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:43.285439 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:43.285666 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.286178 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.286966 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.287185 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.287272 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:43.287579 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:43.287996 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:43.288090 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:43.288222 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 13:54:43.288270 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
I20260926 13:54:43.296197 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:43.291714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.10.236.252:42769)
0926 13:54:43.291911 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:43.291917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:43.292247 (+ 330us) server_negotiation.cc:407] Connection header received
0926 13:54:43.292422 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:43.292425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:43.292505 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:43.292572 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:43.292914 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.293432 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.294738 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.295087 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.295161 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:43.295660 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:43.295742 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:43.295792 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:43.295996 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 13:54:43.296054 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260926 13:54:43.296216 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:43.291692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.10.236.253:41405)
0926 13:54:43.292026 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:43.292031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:43.292209 (+ 178us) server_negotiation.cc:407] Connection header received
0926 13:54:43.292340 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:43.292343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:43.292394 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:43.292470 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:43.292754 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.293550 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.294738 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.295087 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.295161 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:43.295673 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:43.295751 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:43.295796 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:43.295995 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 13:54:43.296052 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260926 13:54:43.297016 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:43.291811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.10.236.254:37411)
0926 13:54:43.292198 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:43.292202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:43.292215 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:54:43.292329 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:43.292332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:43.292496 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:43.292610 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:43.292897 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.293676 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.294867 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:43.295275 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:43.295782 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:43.295930 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:43.296297 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:43.296765 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:43.296801 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:54:43.296859 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":59,"threads_started":1}
I20260926 13:54:43.300033 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 985
I20260926 13:54:43.306550 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 1060
I20260926 13:54:43.312902 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 1135
2026-09-26T13:54:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3172 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:54:43 dist-test-slave-r34j krb5kdc[1351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:54:43 dist-test-slave-r34j krb5kdc[1351](info): set up 2 sockets
Sep 26 13:54:43 dist-test-slave-r34j krb5kdc[1351](info): commencing operation
krb5kdc: starting...
W20260926 13:54:44.364038 11187 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/test-user.keytab.
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:54:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:54:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.236.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
Entry for principal kudu/127.10.236.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab.
I20260926 13:54:44.495234 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:40701
--webserver_interface=127.10.236.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:40701,127.10.236.253:46585,127.10.236.252:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:54:44.610110 1364 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:54:44.610412 1364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:44.610437 1364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:44.610529 1364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:44.614638 1364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:44.614701 1364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:44.614722 1364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:44.614738 1364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:44.614754 1364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:44.620280 1364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:40701,127.10.236.253:46585,127.10.236.252:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.254.keytab
--principal=kudu/127.10.236.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.254:40701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.236.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:44.621429 1364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:44.622505 1364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:54:44.631546 1364 server_base.cc:1034] running on GCE node
W20260926 13:54:44.631644 1370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:44.631611 1369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:44.631517 1372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:44.632396 1364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:44.633690 1364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:44.634876 1364 hybrid_clock.cc:648] HybridClock initialized: now 1790430884634847 us; error 42 us; skew 500 ppm
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:44.638049 1364 init.cc:377] Logged in from keytab as kudu/127.10.236.254@KRBTEST.COM (short username kudu)
I20260926 13:54:44.638991 1364 webserver.cc:460] Webserver started at http://127.10.236.254:33185/ using document root <none> and password file <none>
I20260926 13:54:44.639618 1364 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:44.639693 1364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:44.639930 1364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:44.642014 1364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data/instance:
uuid: "155f80567e984a75853cae28ba4124fd"
format_stamp: "Formatted at 2026-09-26 13:54:44 on dist-test-slave-r34j"
server_key: "69676500859684253ee1fe478a901c72"
server_key_iv: "a50c552cb8e44d0d65ae4108046b2ee2"
server_key_version: "encryptionkey@0"
I20260926 13:54:44.642594 1364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal/instance:
uuid: "155f80567e984a75853cae28ba4124fd"
format_stamp: "Formatted at 2026-09-26 13:54:44 on dist-test-slave-r34j"
server_key: "69676500859684253ee1fe478a901c72"
server_key_iv: "a50c552cb8e44d0d65ae4108046b2ee2"
server_key_version: "encryptionkey@0"
I20260926 13:54:44.646502 1364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:44.649217 1379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:44.650604 1364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:54:44.650836 1364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal
uuid: "155f80567e984a75853cae28ba4124fd"
format_stamp: "Formatted at 2026-09-26 13:54:44 on dist-test-slave-r34j"
server_key: "69676500859684253ee1fe478a901c72"
server_key_iv: "a50c552cb8e44d0d65ae4108046b2ee2"
server_key_version: "encryptionkey@0"
I20260926 13:54:44.650959 1364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:44.664253 1364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:44.667311 1364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:44.667541 1364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:44.674825 1364 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.254:40701
I20260926 13:54:44.675140 1431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.254:40701 every 8 connection(s)
I20260926 13:54:44.675877 1364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/data/info.pb
I20260926 13:54:44.679113 1432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:44.682384 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 1364
I20260926 13:54:44.682518 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-0/wal/instance
I20260926 13:54:44.682847 11187 external_mini_cluster.cc:1428] Setting key 434d4f2aafbcae0f14cbd46da0ba3658
I20260926 13:54:44.684873 1432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.236.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.253@KRBTEST.COM" created.
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:44.702391 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.687804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34992 (local address 127.10.236.254:40701)
0926 13:54:44.688319 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:44.688342 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:44.688582 (+ 240us) server_negotiation.cc:407] Connection header received
0926 13:54:44.689635 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:44.689643 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:44.689945 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:44.690168 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:44.691022 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.692514 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.693740 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.694177 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.694308 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.697373 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:44.697379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:44.697390 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:44.697413 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:44.699091 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.699705 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.699710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.699711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.699841 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.700222 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.700224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.700225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.700710 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:44.700848 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:44.701332 (+ 484us) server_negotiation.cc:299] Negotiation successful
0926 13:54:44.701557 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":331,"thread_start_us":129,"threads_started":1}
I20260926 13:54:44.702394 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.687804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40701 (local address 127.0.0.1:34992)
0926 13:54:44.688411 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.688459 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:44.689303 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:44.690225 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:44.690231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:44.690593 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:44.690855 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.690868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.692721 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.692726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.693615 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.693626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.694425 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.694429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.694510 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.695026 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:44.695041 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:44.697159 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:44.699208 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.699220 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.699234 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.699593 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.699949 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.699956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.699961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.700118 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.700857 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:44.700867 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:44.701051 (+ 184us) client_negotiation.cc:769] Sending connection context
0926 13:54:44.701258 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 13:54:44.701650 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":59,"spinlock_wait_cycles":31616,"thread_start_us":149,"threads_started":1}
I20260926 13:54:44.705186 1432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:44.706053 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.705697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35886)
0926 13:54:44.705899 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.705985 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260926 13:54:44.706324 1381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
I20260926 13:54:44.707376 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.707037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35892)
0926 13:54:44.707190 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.707317 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 13:54:44.707537 1432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:46585: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab.
Entry for principal kudu/127.10.236.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab.
I20260926 13:54:44.716059 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:46585
--webserver_interface=127.10.236.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab
--principal=kudu/127.10.236.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:40701,127.10.236.253:46585,127.10.236.252:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:54:44.756793 1432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } attempt: 1
I20260926 13:54:44.757759 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.757431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35900)
0926 13:54:44.757613 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.757681 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 13:54:44.758544 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.758302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35916)
0926 13:54:44.758435 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.758489 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 13:54:44.758673 1432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:46585: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
I20260926 13:54:44.826867 1432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } attempt: 2
W20260926 13:54:44.826857 1439 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:54:44.827338 1439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:44.827414 1439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:44.827513 1439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 13:54:44.827961 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.827511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35928)
0926 13:54:44.827796 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.827872 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 13:54:44.828919 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.828684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35930)
0926 13:54:44.828798 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.828851 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 13:54:44.829105 1432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.253:46585: Network error: Client connection negotiation failed: client connection to 127.10.236.253:46585: connect: Connection refused (error 111)
W20260926 13:54:44.832361 1439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:44.832417 1439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:44.832437 1439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:44.832490 1439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:44.832530 1439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:44.838054 1439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:40701,127.10.236.253:46585,127.10.236.252:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.253.keytab
--principal=kudu/127.10.236.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.253:46585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.236.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:44.839339 1439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:44.840437 1439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:44.849308 1444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:44.849520 1445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:44.849402 1439 server_base.cc:1034] running on GCE node
W20260926 13:54:44.849313 1447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:44.850170 1439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:44.851353 1439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:44.852528 1439 hybrid_clock.cc:648] HybridClock initialized: now 1790430884852507 us; error 53 us; skew 500 ppm
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:44.855625 1439 init.cc:377] Logged in from keytab as kudu/127.10.236.253@KRBTEST.COM (short username kudu)
I20260926 13:54:44.856540 1439 webserver.cc:460] Webserver started at http://127.10.236.253:45429/ using document root <none> and password file <none>
I20260926 13:54:44.857064 1439 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:44.857122 1439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:44.857293 1439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:44.859324 1439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data/instance:
uuid: "bc9be005e6b54ffc977f6bc0451a5516"
format_stamp: "Formatted at 2026-09-26 13:54:44 on dist-test-slave-r34j"
server_key: "15a529f37ef029c1d42cf10ec4b08772"
server_key_iv: "d85c42f2a2772bdb1c1a3c23a9761e4c"
server_key_version: "encryptionkey@0"
I20260926 13:54:44.859932 1439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal/instance:
uuid: "bc9be005e6b54ffc977f6bc0451a5516"
format_stamp: "Formatted at 2026-09-26 13:54:44 on dist-test-slave-r34j"
server_key: "15a529f37ef029c1d42cf10ec4b08772"
server_key_iv: "d85c42f2a2772bdb1c1a3c23a9761e4c"
server_key_version: "encryptionkey@0"
I20260926 13:54:44.863762 1439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 13:54:44.866272 1454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:44.867668 1439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 13:54:44.867774 1439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal
uuid: "bc9be005e6b54ffc977f6bc0451a5516"
format_stamp: "Formatted at 2026-09-26 13:54:44 on dist-test-slave-r34j"
server_key: "15a529f37ef029c1d42cf10ec4b08772"
server_key_iv: "d85c42f2a2772bdb1c1a3c23a9761e4c"
server_key_version: "encryptionkey@0"
I20260926 13:54:44.867916 1439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:44.891654 1439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:44.895116 1439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:44.895345 1439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:44.902650 1439 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.253:46585
I20260926 13:54:44.903004 1506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.253:46585 every 8 connection(s)
I20260926 13:54:44.903733 1439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/data/info.pb
I20260926 13:54:44.906555 1507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:44.910388 1507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:44.913398 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 1439
I20260926 13:54:44.913539 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-1/wal/instance
I20260926 13:54:44.913887 11187 external_mini_cluster.cc:1428] Setting key 3f8f03d954da03ebfe06db24ee9aad58
I20260926 13:54:44.920312 1432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } attempt: 3
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.236.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.236.252@KRBTEST.COM" created.
I20260926 13:54:44.926210 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.912369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.10.236.254:40701)
0926 13:54:44.912520 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:44.912524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:44.913518 (+ 994us) server_negotiation.cc:407] Connection header received
0926 13:54:44.914984 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:44.914987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:44.915043 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:44.915119 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:44.916551 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.917092 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.918386 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.918629 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.918760 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.921627 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:44.921631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:44.921634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:44.921661 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:44.923463 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.924121 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.924124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.924126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.924211 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.924595 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.924598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.924600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.924852 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:44.924964 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:44.925731 (+ 767us) server_negotiation.cc:299] Negotiation successful
0926 13:54:44.925944 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:54:44.928838 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.912751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40701 (local address 127.0.0.1:35004)
0926 13:54:44.913342 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.913403 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:44.914734 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:44.915351 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:44.915362 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:44.915895 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:44.916388 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.916401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.917243 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.917247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.918253 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.918263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.918801 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.918806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.918996 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.919668 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:44.919686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:44.921436 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:44.923631 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.923637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.923656 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.924021 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.924318 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.924325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.924330 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.924501 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.924996 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:44.925004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:44.925369 (+ 365us) client_negotiation.cc:769] Sending connection context
0926 13:54:44.925596 (+ 227us) client_negotiation.cc:240] Negotiation successful
0926 13:54:44.926006 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":131,"threads_started":1}
I20260926 13:54:44.930501 1507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:44.932780 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.920812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35938)
0926 13:54:44.920992 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.921007 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:44.921107 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:44.921747 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:44.921750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:44.921980 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:44.922135 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.922143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.923342 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.923347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.924229 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.924238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.925039 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.925043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.925256 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.925780 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:44.925795 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:44.927973 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:44.930859 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.930862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.930865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.931137 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.931513 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.931520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.931522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.931632 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.932267 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:44.932272 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:44.932386 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 13:54:44.932486 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 13:54:44.932618 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 13:54:44.933017 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.920936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.10.236.253:46585)
0926 13:54:44.921290 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:44.921303 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:44.921329 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:54:44.921380 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:44.921387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:44.921612 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:44.921758 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:44.922272 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.923220 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.924380 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.924888 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.924997 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.928136 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:44.928144 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:44.928158 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:44.928182 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:44.930723 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.931265 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.931271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.931275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.931380 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.931779 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.931785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.931790 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.932144 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:44.932302 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:44.932626 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 13:54:44.932798 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":126,"threads_started":1}
I20260926 13:54:44.935705 1432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:44.936468 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.936182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:58998)
0926 13:54:44.936315 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.936363 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 13:54:44.937160 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.936865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59006)
0926 13:54:44.937047 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.937099 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260926 13:54:44.937372 1432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:40671: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Sep 26 13:54:44 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:54:44.942236 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.931054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35950)
0926 13:54:44.931170 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.931186 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:44.931257 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:44.931741 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:44.931744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:44.931912 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:44.931990 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.931995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.933007 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.933010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.933791 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:44.933804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.934507 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.934510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:44.934594 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.935062 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:44.935075 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:44.938743 (+ 3668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:44.940675 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.940678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.940679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.940880 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.941221 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:44.941224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:44.941225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:44.941323 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:44.941758 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:44.941761 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:44.941895 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 13:54:44.942004 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 13:54:44.942122 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:54:44.942601 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.931223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.10.236.253:46585)
0926 13:54:44.931546 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:44.931554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:44.931565 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:54:44.931602 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:44.931604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:44.931637 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:44.931733 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:44.932097 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.932902 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.933944 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:44.934380 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:44.934801 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:44.938891 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:44.938895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:44.938896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:44.938918 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:44.940577 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.940966 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.940969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.940970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.941080 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:44.941440 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:44.941443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:44.941444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:44.941633 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:44.941768 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:44.942150 (+ 382us) server_negotiation.cc:299] Negotiation successful
0926 13:54:44.942364 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":111,"threads_started":1}
I20260926 13:54:44.943550 1507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:44.944296 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.943969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59008)
0926 13:54:44.944133 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.944239 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 13:54:44.944542 1455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
I20260926 13:54:44.945544 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.945294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59014)
0926 13:54:44.945436 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.945491 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 13:54:44.945695 1507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:40671: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.236.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab.
Entry for principal kudu/127.10.236.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab.
I20260926 13:54:44.948043 11187 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
/tmp/dist-test-taskMS3Khi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:40671
--webserver_interface=127.10.236.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab
--principal=kudu/127.10.236.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.236.212:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.236.254:40701,127.10.236.253:46585,127.10.236.252:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:54:44.968544 1432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } attempt: 1
I20260926 13:54:44.969514 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.969180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59030)
0926 13:54:44.969365 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.969430 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 13:54:44.970263 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.970065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59040)
0926 13:54:44.970164 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.970208 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 13:54:44.970413 1432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:40671: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
I20260926 13:54:44.995009 1507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } attempt: 1
I20260926 13:54:44.996099 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.995743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59048)
0926 13:54:44.995925 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.996001 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 13:54:44.996915 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:44.996687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59052)
0926 13:54:44.996797 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:44.996853 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:54:44.997051 1507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:40671: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
I20260926 13:54:45.045611 1432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } attempt: 2
I20260926 13:54:45.046597 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.046261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59068)
0926 13:54:45.046448 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.046519 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 13:54:45.047636 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.047328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59070)
0926 13:54:45.047465 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.047578 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 13:54:45.047870 1432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:40671: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
I20260926 13:54:45.065217 1507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } attempt: 2
I20260926 13:54:45.066090 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.065761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59080)
0926 13:54:45.065927 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.065997 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 13:54:45.066810 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.066583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59092)
0926 13:54:45.066686 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.066737 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 13:54:45.066954 1507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.236.252:40671: Network error: Client connection negotiation failed: client connection to 127.10.236.252:40671: connect: Connection refused (error 111)
W20260926 13:54:45.070263 1515 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:54:45.070533 1515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:54:45.070559 1515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:54:45.070597 1515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:54:45.075135 1515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:54:45.075204 1515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:54:45.075224 1515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:54:45.075242 1515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:54:45.075258 1515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:54:45.080883 1515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.236.212:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.236.254:40701,127.10.236.253:46585,127.10.236.252:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/krb5kdc/kudu_127.10.236.252.keytab
--principal=kudu/127.10.236.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.236.252:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.236.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 12:47:21 UTC on ccycloud.quasar-ftxjdu.root.comops.site
build id 4673841
I20260926 13:54:45.081943 1515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:54:45.082943 1515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:54:45.091892 1523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:45.091905 1521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:54:45.091905 1520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:54:45.092374 1515 server_base.cc:1034] running on GCE node
I20260926 13:54:45.092913 1515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:54:45.093994 1515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:54:45.095202 1515 hybrid_clock.cc:648] HybridClock initialized: now 1790430885095186 us; error 37 us; skew 500 ppm
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430885, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:54:45.098137 1515 init.cc:377] Logged in from keytab as kudu/127.10.236.252@KRBTEST.COM (short username kudu)
I20260926 13:54:45.099104 1515 webserver.cc:460] Webserver started at http://127.10.236.252:44735/ using document root <none> and password file <none>
I20260926 13:54:45.099776 1515 fs_manager.cc:359] Metadata directory not provided
I20260926 13:54:45.099870 1515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:54:45.100103 1515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:54:45.102164 1515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data/instance:
uuid: "d77fcd021224404fbc9c210f1b799f55"
format_stamp: "Formatted at 2026-09-26 13:54:45 on dist-test-slave-r34j"
server_key: "f1780790094be0b83ea6beef987c466a"
server_key_iv: "6240ef5b51704be87f36c9ca099b86a8"
server_key_version: "encryptionkey@0"
I20260926 13:54:45.102725 1515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal/instance:
uuid: "d77fcd021224404fbc9c210f1b799f55"
format_stamp: "Formatted at 2026-09-26 13:54:45 on dist-test-slave-r34j"
server_key: "f1780790094be0b83ea6beef987c466a"
server_key_iv: "6240ef5b51704be87f36c9ca099b86a8"
server_key_version: "encryptionkey@0"
I20260926 13:54:45.106554 1515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 13:54:45.109505 1530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:54:45.110991 1515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:54:45.111120 1515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data,/tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal
uuid: "d77fcd021224404fbc9c210f1b799f55"
format_stamp: "Formatted at 2026-09-26 13:54:45 on dist-test-slave-r34j"
server_key: "f1780790094be0b83ea6beef987c466a"
server_key_iv: "6240ef5b51704be87f36c9ca099b86a8"
server_key_version: "encryptionkey@0"
I20260926 13:54:45.111227 1515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:54:45.127231 1515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:54:45.130993 1515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:54:45.131166 1515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:54:45.139382 1515 rpc_server.cc:307] RPC server started. Bound to: 127.10.236.252:40671
I20260926 13:54:45.139679 1582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.236.252:40671 every 8 connection(s)
I20260926 13:54:45.140726 1515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/data/info.pb
I20260926 13:54:45.143929 1583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:54:45.145689 11187 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu as pid 1515
I20260926 13:54:45.145844 11187 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMS3Khi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790430594225530-11187-0/minicluster-data/master-2/wal/instance
I20260926 13:54:45.146179 11187 external_mini_cluster.cc:1428] Setting key db522dba2361ca92148c94c5b2566c40
I20260926 13:54:45.147080 1432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } attempt: 3
I20260926 13:54:45.150055 1583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.254@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430885, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:45.158120 1507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } attempt: 3
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:54:45.163872 1436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.147786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59108)
0926 13:54:45.148096 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.148114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:45.148202 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:45.150496 (+ 2294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:45.150500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:45.150752 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:45.150896 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.150906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.152961 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.152967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.153826 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.153837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.154841 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.154845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.154926 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.155629 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:45.155644 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:45.157383 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:45.161606 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.161609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.161611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.161881 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.162303 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.162308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.162311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.162430 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.163289 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:45.163293 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:45.163395 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 13:54:45.163515 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 13:54:45.163677 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260926 13:54:45.164925 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.152019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.254:40701 (local address 127.0.0.1:35006)
0926 13:54:45.152477 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.152529 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:45.152709 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:45.153021 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:45.153026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:45.153410 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:45.153515 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.153521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.154425 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.154429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.155385 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.155396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.156150 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.156157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.156275 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.156800 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:45.156834 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:45.159286 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:45.161606 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.161613 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.161624 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.161927 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.162237 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.162241 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.162245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.162425 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.163069 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:45.163078 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:45.163260 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 13:54:45.163452 (+ 192us) client_negotiation.cc:240] Negotiation successful
0926 13:54:45.163826 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":91,"threads_started":1}
I20260926 13:54:45.164930 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.148575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.10.236.252:40671)
0926 13:54:45.148851 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.148858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.148870 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:54:45.149458 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.149477 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.150296 (+ 819us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.151034 (+ 738us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.151181 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.152816 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.153954 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.154694 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.154899 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.157553 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.157560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.157574 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.157599 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.161475 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.162019 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.162024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.162026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.162173 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.162575 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.162579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.162580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.163137 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.164247 (+ 1110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.164356 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.164516 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":200,"spinlock_wait_cycles":4352,"thread_start_us":89,"threads_started":1}
I20260926 13:54:45.165653 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.152264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.10.236.254:40701)
0926 13:54:45.152432 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.152436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.152600 (+ 164us) server_negotiation.cc:407] Connection header received
0926 13:54:45.152827 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.152831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.152897 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.152974 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.153635 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.154285 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.155557 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.155880 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.155951 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.159663 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.159671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.159677 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.159713 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.161497 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.162029 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.162033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.162035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.162132 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.162541 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.162546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.162550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.162906 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.163012 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.165356 (+ 2344us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.165486 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":98}
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.253@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:54:45.166453 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.148157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59106 (local address 127.10.236.252:40671)
0926 13:54:45.148659 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.148683 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.148728 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:54:45.149484 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.149488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.149896 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.150304 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.151456 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.152846 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.154044 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.154711 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.155672 (+ 961us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.160105 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.160110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.160114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.160143 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.163205 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.163746 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.163750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.163752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.163846 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.164439 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.164442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.164444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.164631 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.164775 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.166158 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.166279 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":322,"spinlock_wait_cycles":10496,"thread_start_us":137,"threads_started":1}
I20260926 13:54:45.166857 1583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:45.170889 1432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd: Bootstrap starting.
I20260926 13:54:45.171178 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.159002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.10.236.252:40671)
0926 13:54:45.159958 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.159970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.160367 (+ 397us) server_negotiation.cc:407] Connection header received
0926 13:54:45.160410 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.160415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.160467 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.160555 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.161109 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.162488 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.163775 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.164134 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.166765 (+ 2631us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.166895 (+ 130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.166899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.166902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.167181 (+ 279us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.169192 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.169779 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.169782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.169784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.169925 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.170281 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.170285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.170286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.170499 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.170607 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.170915 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.171019 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":702,"threads_started":1}
I20260926 13:54:45.171183 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.159920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59110)
0926 13:54:45.160146 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.160163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:45.160264 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:45.160617 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:45.160623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:45.160852 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:45.160972 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.160979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.162700 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.162704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.163605 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.163619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.164285 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.164289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.164385 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.164910 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:45.164930 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:45.166598 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:45.169338 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.169342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.169344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.169620 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.170049 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.170053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.170055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.170154 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.170611 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:45.170616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:45.170722 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 13:54:45.170847 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 13:54:45.170985 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430885, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:54:45.174301 1432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:45.174575 1507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: Bootstrap starting.
I20260926 13:54:45.175269 1432 log.cc:824] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:45.176841 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.167649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.10.236.253:46585)
0926 13:54:45.167781 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.167819 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.167854 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:54:45.167987 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.167990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.168033 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.168128 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.168683 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.169264 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.170023 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.170277 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.170394 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.172936 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.172939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.172942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.172968 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.174881 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.175358 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.175362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.175364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.175456 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.175846 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.175850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.175852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.176141 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.176266 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.176557 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.176680 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:54:45.176841 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.167572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.253:46585 (local address 127.0.0.1:35964)
0926 13:54:45.167791 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.167811 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:45.167898 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:45.168249 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:45.168253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:45.168477 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:45.168564 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.168569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.169371 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.169378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.169905 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.169912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.170393 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.170396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.170508 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.171323 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:45.171341 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:45.172812 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:45.175005 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.175008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.175010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.175237 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.175593 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.175596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.175598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.175700 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.176257 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:45.176260 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:45.176373 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 13:54:45.176502 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 13:54:45.176652 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260926 13:54:45.177141 1507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: Neither blocks nor log segments found. Creating new log.
I20260926 13:54:45.177618 1432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd: No bootstrap required, opened a new log
I20260926 13:54:45.177575 1583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:54:45.177938 1507 log.cc:824] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:45.180959 1507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: No bootstrap required, opened a new log
I20260926 13:54:45.181806 1432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
I20260926 13:54:45.182029 1432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155f80567e984a75853cae28ba4124fd, State: Initialized, Role: FOLLOWER
I20260926 13:54:45.182348 1432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430885, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.236.252@KRBTEST.COM for kudu/127.10.236.252@KRBTEST.COM
I20260926 13:54:45.183758 1432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:45.185243 1507 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
I20260926 13:54:45.185598 1507 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9be005e6b54ffc977f6bc0451a5516, State: Initialized, Role: FOLLOWER
I20260926 13:54:45.185966 1507 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.254@KRBTEST.COM
I20260926 13:54:45.187507 1507 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:45.188126 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.178061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.236.252:40671 (local address 127.0.0.1:59118)
0926 13:54:45.178160 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 13:54:45.178177 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:54:45.178255 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:54:45.178529 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:54:45.178532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:54:45.178714 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:54:45.178778 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.178784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.179867 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.179870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.180795 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:54:45.180805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.181506 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.181511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:54:45.181590 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.182114 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:54:45.182130 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:54:45.183870 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:54:45.186250 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.186253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.186256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.186508 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.186821 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:54:45.186824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:54:45.186826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:54:45.186941 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:54:45.187507 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:54:45.187513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:54:45.187629 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 13:54:45.187752 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 13:54:45.187917 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 13:54:45.188787 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.178230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59118 (local address 127.10.236.252:40671)
0926 13:54:45.178342 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.178346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.178357 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:54:45.178385 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.178387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.178430 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.178520 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.178894 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.179740 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.180926 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.181295 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.181399 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.184040 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.184045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.184049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.184077 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.186093 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.186623 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.186626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.186628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.186710 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.187071 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.187077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.187083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.187328 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.187438 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.188486 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.188619 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:54:45.191942 1583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: Bootstrap starting.
I20260926 13:54:45.188875 1364 master_runner.cc:186] Error getting master registration for 127.10.236.253:46585: OK, instance_id { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" instance_seqno: 1790430884900588 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:54:45.192483 1364 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 13:54:45.193851 1615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:45.193941 1615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:45.191380 1592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.194192 1592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.194864 1583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: Neither blocks nor log segments found. Creating new log.
W20260926 13:54:45.195187 1603 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:45.195312 1603 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:45.195847 1583 log.cc:824] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: Log is configured to *not* fsync() on all Append() calls
I20260926 13:54:45.198823 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.172419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.10.236.254:40701)
0926 13:54:45.178981 (+ 6562us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.178985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.179000 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:54:45.179038 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.179041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.179080 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.179166 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.180071 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.180897 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.182480 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.182895 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.183006 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.187173 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.187177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.187180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.187207 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.189188 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.196752 (+ 7564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.196757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.196762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.196879 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.197246 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.197251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.197253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.197508 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.197740 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.198499 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.198643 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6462}
I20260926 13:54:45.197359 1439 master_runner.cc:186] Error getting master registration for 127.10.236.252:40671: OK, instance_id { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" instance_seqno: 1790430885136931 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:54:45.200763 1439 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:54:45.200987 1604 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.201116 1604 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.205734 1583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: No bootstrap required, opened a new log
Sep 26 13:54:45 dist-test-slave-r34j krb5kdc[1351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790430884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.236.253@KRBTEST.COM
I20260926 13:54:45.210671 1583 raft_consensus.cc:348] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
I20260926 13:54:45.210971 1583 raft_consensus.cc:729] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77fcd021224404fbc9c210f1b799f55, State: Initialized, Role: FOLLOWER
I20260926 13:54:45.211339 1583 consensus_queue.cc:260] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
I20260926 13:54:45.212527 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.200127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.10.236.253:46585)
0926 13:54:45.200252 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.200256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.200524 (+ 268us) server_negotiation.cc:407] Connection header received
0926 13:54:45.200731 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.200735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.200787 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.200844 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.201816 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.202738 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.204115 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.204504 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.204594 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.208175 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.208180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.208184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.208215 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.210222 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.210868 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.210874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.210877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.210980 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.211334 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.211338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.211340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.211617 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.211797 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.212200 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.212343 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:54:45.212816 1583 sys_catalog.cc:564] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:54:45.215091 1619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.215657 1619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.217039 11187 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:54:45.218407 1406 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "155f80567e984a75853cae28ba4124fd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35016
I20260926 13:54:45.218668 1406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 13:54:45.218842 1406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:45.220335 1406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
W20260926 13:54:45.221123 1630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:54:45.221203 1630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:54:45.221846 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "155f80567e984a75853cae28ba4124fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d77fcd021224404fbc9c210f1b799f55"
I20260926 13:54:45.221849 1481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "155f80567e984a75853cae28ba4124fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bc9be005e6b54ffc977f6bc0451a5516"
I20260926 13:54:45.222110 1557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:45.222110 1481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:54:45.223575 1481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 155f80567e984a75853cae28ba4124fd in term 1.
I20260926 13:54:45.223606 1557 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 155f80567e984a75853cae28ba4124fd in term 1.
I20260926 13:54:45.224053 1381 leader_election.cc:302] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 155f80567e984a75853cae28ba4124fd, bc9be005e6b54ffc977f6bc0451a5516, d77fcd021224404fbc9c210f1b799f55; no voters:
I20260926 13:54:45.224436 1592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:54:45.224711 1592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [term 1 LEADER]: Becoming Leader. State: Replica: 155f80567e984a75853cae28ba4124fd, State: Running, Role: LEADER
I20260926 13:54:45.225107 1592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } }
I20260926 13:54:45.225898 1515 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260926 13:54:45.226022 1515 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:54:45.226100 1406 leader_election.cc:288] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [CANDIDATE]: Term 1 election: Requested vote from peers bc9be005e6b54ffc977f6bc0451a5516 (127.10.236.253:46585), d77fcd021224404fbc9c210f1b799f55 (127.10.236.252:40671)
I20260926 13:54:45.227061 1592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 155f80567e984a75853cae28ba4124fd. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.227192 1592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:45.233594 1634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:54:45.234663 1634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:54:45.260942 1557 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [term 1 FOLLOWER]: Refusing update from remote peer 155f80567e984a75853cae28ba4124fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:45.261729 1592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:45.266654 1619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 155f80567e984a75853cae28ba4124fd. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.266826 1619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.268239 1481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [term 1 FOLLOWER]: Refusing update from remote peer 155f80567e984a75853cae28ba4124fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:54:45.268903 1632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:54:45.274047 1604 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 155f80567e984a75853cae28ba4124fd. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.274202 1604 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.281772 1592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.281944 1592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:45.283205 1619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.283367 1619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.283658 1640 mvcc.cc:205] Tried to move back new op lower bound from 7333604905955192832 to 7333604905885900800. Current Snapshot: MvccSnapshot[applied={T|T < 7333604905955192832}]
I20260926 13:54:45.284296 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.232075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.10.236.254:40701)
0926 13:54:45.232199 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.232204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.237876 (+ 5672us) server_negotiation.cc:407] Connection header received
0926 13:54:45.238393 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.238399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.238454 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.238511 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.245719 (+ 7208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.246702 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.253812 (+ 7110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.254234 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.254322 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.262524 (+ 8202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.262528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.262531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.262570 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.264474 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.276995 (+ 12521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.277000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.277002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.277113 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.278222 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.278226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.278228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.278494 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.278626 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.283964 (+ 5338us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.284124 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:54:45.287344 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.232764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.10.236.253:46585)
0926 13:54:45.232882 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.232886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.238081 (+ 5195us) server_negotiation.cc:407] Connection header received
0926 13:54:45.239657 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.239661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.239730 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.239808 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.246865 (+ 7057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.247790 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.252482 (+ 4692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.252899 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.252988 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.265096 (+ 12108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.265100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.265104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.265139 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.267052 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.277962 (+ 10910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.277967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.277969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.278075 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.279745 (+ 1670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.279749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.279751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.280013 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.280139 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.287020 (+ 6881us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.287168 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:54:45.289304 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:45.233194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.10.236.252:40671)
0926 13:54:45.233311 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.233315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.240121 (+ 6806us) server_negotiation.cc:407] Connection header received
0926 13:54:45.244319 (+ 4198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.244323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.244381 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.244437 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.250312 (+ 5875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.251194 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.255056 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:45.255442 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:45.255556 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:45.274515 (+ 18959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:54:45.274520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:54:45.274524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:54:45.274566 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:54:45.276593 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.279187 (+ 2594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.279191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.279193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.279291 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:54:45.286136 (+ 6845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:54:45.286142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:54:45.286145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:54:45.286408 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:54:45.286566 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:45.289014 (+ 2448us) server_negotiation.cc:299] Negotiation successful
0926 13:54:45.289149 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:54:45.299233 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.299420 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd [sys.catalog]: This master's current role is: LEADER
I20260926 13:54:45.300815 1604 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "155f80567e984a75853cae28ba4124fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155f80567e984a75853cae28ba4124fd" member_type: VOTER last_known_addr { host: "127.10.236.254" port: 40701 } } peers { permanent_uuid: "bc9be005e6b54ffc977f6bc0451a5516" member_type: VOTER last_known_addr { host: "127.10.236.253" port: 46585 } } peers { permanent_uuid: "d77fcd021224404fbc9c210f1b799f55" member_type: VOTER last_known_addr { host: "127.10.236.252" port: 40671 } } }
I20260926 13:54:45.300956 1604 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:54:45.302150 1645 mvcc.cc:205] Tried to move back new op lower bound from 7333604905955192832 to 7333604905885900800. Current Snapshot: MvccSnapshot[applied={T|T < 7333604905955192832}]
I20260926 13:54:45.302415 1634 catalog_manager.cc:1347] Generated new cluster ID: c5d4ce7e68a549c29b4264646e23ec70
I20260926 13:54:45.302481 1634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:54:45.341786 1634 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:54:45.343248 1634 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:54:45.389541 1634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 155f80567e984a75853cae28ba4124fd: Generated new TSK 0
I20260926 13:54:45.390551 1634 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:54:45.426750 1435 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.424982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.10.236.254:40701)
0926 13:54:45.425140 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.425147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.425600 (+ 453us) server_negotiation.cc:407] Connection header received
0926 13:54:45.425810 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.425816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.425886 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.425941 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.426605 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35024: BlockingRecv error: recv got EOF from 127.0.0.1:35024 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 13:54:45.427778 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.425976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.10.236.253:46585)
0926 13:54:45.426202 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.426206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.426511 (+ 305us) server_negotiation.cc:407] Connection header received
0926 13:54:45.426687 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.426691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.426738 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.426787 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.427667 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35992: BlockingRecv error: recv got EOF from 127.0.0.1:35992 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
W20260926 13:54:45.428020 1589 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.426086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.10.236.252:40671)
0926 13:54:45.426282 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.426286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.427154 (+ 868us) server_negotiation.cc:407] Connection header received
0926 13:54:45.427306 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.427309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.427356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.427407 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.427905 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59138: BlockingRecv error: recv got EOF from 127.0.0.1:59138 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
W20260926 13:54:45.431331 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.430330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36006 (local address 127.10.236.253:46585)
0926 13:54:45.430427 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.430431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.430565 (+ 134us) server_negotiation.cc:407] Connection header received
0926 13:54:45.430691 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.430694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.430771 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.430866 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.431234 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36006: BlockingRecv error: recv got EOF from 127.0.0.1:36006 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260926 13:54:45.435810 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.434888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.10.236.253:46585)
0926 13:54:45.435031 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.435035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.435048 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:54:45.435095 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.435098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.435146 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.435228 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.435709 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36014: BlockingRecv error: recv got EOF from 127.0.0.1:36014 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 13:54:45.441268 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.439890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.10.236.253:46585)
0926 13:54:45.440035 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.440040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.440355 (+ 315us) server_negotiation.cc:407] Connection header received
0926 13:54:45.440455 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.440457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.440503 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.440569 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.441139 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36030: BlockingRecv error: recv got EOF from 127.0.0.1:36030 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260926 13:54:45.448516 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.447216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.10.236.253:46585)
0926 13:54:45.447392 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.447396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.447570 (+ 174us) server_negotiation.cc:407] Connection header received
0926 13:54:45.447712 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.447715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.447761 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.447835 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.448394 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36046: BlockingRecv error: recv got EOF from 127.0.0.1:36046 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260926 13:54:45.459484 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.458261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.10.236.253:46585)
0926 13:54:45.458415 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.458418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.458569 (+ 151us) server_negotiation.cc:407] Connection header received
0926 13:54:45.458682 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.458685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.458729 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.458810 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.459325 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36050: BlockingRecv error: recv got EOF from 127.0.0.1:36050 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260926 13:54:45.479014 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.477820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.10.236.253:46585)
0926 13:54:45.477975 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.477979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.478174 (+ 195us) server_negotiation.cc:407] Connection header received
0926 13:54:45.478336 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.478338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.478392 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.478484 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.478918 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36060: BlockingRecv error: recv got EOF from 127.0.0.1:36060 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 13:54:45.514993 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.513324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.10.236.253:46585)
0926 13:54:45.513481 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.513484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.513650 (+ 166us) server_negotiation.cc:407] Connection header received
0926 13:54:45.513738 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.513740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.513800 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.513905 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.514847 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36062: BlockingRecv error: recv got EOF from 127.0.0.1:36062 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 13:54:45.582517 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.581265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.10.236.253:46585)
0926 13:54:45.581460 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.581464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.581602 (+ 138us) server_negotiation.cc:407] Connection header received
0926 13:54:45.581673 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.581676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.581733 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.581826 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.582381 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36072: BlockingRecv error: recv got EOF from 127.0.0.1:36072 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260926 13:54:45.714965 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.713469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.10.236.253:46585)
0926 13:54:45.713661 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.713665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.713842 (+ 177us) server_negotiation.cc:407] Connection header received
0926 13:54:45.714021 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.714024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.714079 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.714161 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.714836 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36084: BlockingRecv error: recv got EOF from 127.0.0.1:36084 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260926 13:54:45.974696 1513 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:54:45.973695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.10.236.253:46585)
0926 13:54:45.973837 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:45.973841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:45.973856 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:54:45.973934 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:45.973937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:45.973984 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:45.974068 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:54:45.974594 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36088: BlockingRecv error: recv got EOF from 127.0.0.1:36088 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 13:54:46.195353 1615 catalog_manager.cc:1259] Loaded cluster ID: c5d4ce7e68a549c29b4264646e23ec70
I20260926 13:54:46.195448 1615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: loading cluster ID for follower catalog manager: success
I20260926 13:54:46.197324 1615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: acquiring CA information for follower catalog manager: success
I20260926 13:54:46.198589 1615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc9be005e6b54ffc977f6bc0451a5516: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:54:46.222644 1630 catalog_manager.cc:1259] Loaded cluster ID: c5d4ce7e68a549c29b4264646e23ec70
I20260926 13:54:46.222723 1630 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: loading cluster ID for follower catalog manager: success
I20260926 13:54:46.224562 1630 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: acquiring CA information for follower catalog manager: success
I20260926 13:54:46.225764 1630 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d77fcd021224404fbc9c210f1b799f55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:54:46.493928 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:46.489040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.10.236.253:46585)
0926 13:54:46.489293 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:46.489297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:46.489502 (+ 205us) server_negotiation.cc:407] Connection header received
0926 13:54:46.489569 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:46.489571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:46.489703 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:46.489797 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:46.490328 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.490999 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.492158 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.492439 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.492577 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:46.492963 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:46.493450 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:46.493603 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:46.493671 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 13:54:46.493740 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260926 13:54:46.501674 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:46.497505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59154 (local address 127.10.236.252:40671)
0926 13:54:46.497774 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:46.497778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:46.497793 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:54:46.497957 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:46.497959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:46.498078 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:46.498178 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:46.498530 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.499056 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.499973 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.500193 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.500286 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:46.500611 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:46.501143 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:46.501234 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:46.501390 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 13:54:46.501487 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260926 13:54:46.509405 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:46.505378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36100 (local address 127.10.236.253:46585)
0926 13:54:46.505506 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:46.505512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:46.505716 (+ 204us) server_negotiation.cc:407] Connection header received
0926 13:54:46.505907 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:46.505910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:46.505967 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:46.506057 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:46.506317 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.507131 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.508282 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.508522 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.508638 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:46.508902 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:46.508988 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:46.509068 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:46.509203 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 13:54:46.509261 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:54:46.509675 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:46.505135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59156 (local address 127.10.236.252:40671)
0926 13:54:46.505278 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:46.505283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:46.505551 (+ 268us) server_negotiation.cc:407] Connection header received
0926 13:54:46.505723 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:46.505726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:46.505781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:46.505879 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:46.506375 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.506897 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.508236 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.508563 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.508649 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:46.508960 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:46.509130 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:46.509198 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:46.509466 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 13:54:46.509520 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:54:46.510166 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:54:46.505004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.10.236.254:40701)
0926 13:54:46.505312 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0926 13:54:46.505317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:54:46.505571 (+ 254us) server_negotiation.cc:407] Connection header received
0926 13:54:46.505683 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:54:46.505689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:54:46.505870 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:54:46.505999 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:54:46.506229 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.507130 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.508170 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:54:46.508517 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:54:46.508648 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:54:46.509217 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:54:46.509584 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:54:46.509683 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:54:46.509977 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 13:54:46.510041 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":94,"threads_started":1}
I20260926 13:54:46.513273 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 1364
I20260926 13:54:46.520670 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 1439
I20260926 13:54:46.526530 11187 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMS3Khi/build/debug/bin/kudu with pid 1515
2026-09-26T13:54:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3213 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6386 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292292 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST